-- Logs begin at Fri 2022-09-16 18:37:33 UTC, end at Fri 2022-09-16 19:19:46 UTC. -- Sep 16 19:03:47.241794 dv-cinder-803773-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 16 19:03:47.250452 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 16 19:03:47 2022] *** Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: nodename: dv-cinder-803773-5-iscsi Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: machine: x86_64 Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: clock source: unix Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: pcre jit disabled Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: detected number of CPU cores: 4 Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: current working directory: / Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: detected binary path: /usr/bin/uwsgi-core Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: your processes number limit is 47680 Sep 16 19:03:47.252479 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: your memory page size is 4096 bytes Sep 16 19:03:47.253182 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: detected max file descriptor number: 2048 Sep 16 19:03:47.253182 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: lock engine: pthread robust mutexes Sep 16 19:03:47.253182 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: thunder lock: enabled Sep 16 19:03:47.253182 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 16 19:03:47.253182 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 16 19:03:47.292241 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: Python main interpreter initialized at 0x557b66fb99c0 Sep 16 19:03:47.292241 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: python threads support enabled Sep 16 19:03:47.292241 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: your server socket listen backlog is limited to 100 connections Sep 16 19:03:47.292241 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: your mercy for graceful operations on workers is 90 seconds Sep 16 19:03:47.292577 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: mapped 403077 bytes (393 KB) for 2 cores Sep 16 19:03:47.292657 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: *** Operational MODE: preforking *** Sep 16 19:03:47.292915 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: *** uWSGI is running in multiple interpreter mode *** Sep 16 19:03:47.292915 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: spawned uWSGI master process (pid: 81388) Sep 16 19:03:47.293109 dv-cinder-803773-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 16 19:03:47.293673 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: spawned uWSGI worker 1 (pid: 81399, cores: 1) Sep 16 19:03:47.294851 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: spawned uWSGI worker 2 (pid: 81400, cores: 1) Sep 16 19:03:47.294851 dv-cinder-803773-5-iscsi devstack@keystone.service[81388]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 16 19:03:51.560101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 19:03:51.560101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Sep 16 19:03:51.560899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 19:03:51.560899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Sep 16 19:03:51.561025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 19:03:51.561025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Sep 16 19:03:51.561025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 19:03:51.561025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Sep 16 19:03:51.610099 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 16 19:03:51.610252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 16 19:03:51.610341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 16 19:03:51.610425 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 16 19:03:51.610501 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] command line args: [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 16 19:03:51.610575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] command line args: [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 16 19:03:51.610719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 16 19:03:51.610802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 16 19:03:51.610899 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ================================================================================ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 16 19:03:51.610981 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ================================================================================ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 16 19:03:51.611149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_dir = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_dir = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_source = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_source = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.611962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] debug = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.612044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] debug = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.612156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.612958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613186 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613285 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613574 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.613978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_config_append = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614169 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_config_append = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_dir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614557 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_dir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614740 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614842 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.614956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615878 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.615963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616154 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616489 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.616953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617082 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617205 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617583 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.617945 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618080 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618367 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] publish_errors = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] publish_errors = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.618998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619393 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.619925 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620045 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620181 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] standard_threads = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] standard_threads = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620564 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620685 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620814 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.620942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_journal = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_journal = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_json = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_json = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621752 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_stderr = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.621885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_stderr = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.622009 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_syslog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.622143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_syslog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.622274 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.622394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 19:03:51.622512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.622633 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.622759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.622891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623527 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.623917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624639 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624777 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.624928 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625003 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625269 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625655 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625875 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.625950 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626092 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626515 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626588 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626734 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.626885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627408 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.627899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.628571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.628660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.628813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.628889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629057 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629595 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.629999 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630312 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630478 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630567 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.630943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631736 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.631965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632119 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632577 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.632892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633348 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.633968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.634899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635386 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.635934 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636306 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636624 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.636855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637712 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.637949 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.638931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639007 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.639967 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640583 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.640976 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641595 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641851 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.641975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642099 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642740 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.642988 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643112 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.643897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644097 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.644877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645635 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.645893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646150 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646529 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646783 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.646909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647032 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647315 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647571 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.647984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.external = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.external = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.648890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.password = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649274 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.token = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.password = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.token = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.649914 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650165 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650400 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650781 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.650905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.651905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652162 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.652918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653169 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653294 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.653799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654537 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654808 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.654934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655616 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.655866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656000 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656131 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.656919 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657559 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.657919 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658310 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.658831 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.659952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660067 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660166 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660271 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660391 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660833 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.660971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661489 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661489 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661851 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.661927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662468 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.662958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663096 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663418 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.663980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664620 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.664907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665020 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665868 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.665980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666651 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.666876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667391 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667618 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667776 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.667903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.668974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669874 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.669951 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670344 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.670895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671548 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.671869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672265 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.672979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673293 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.673922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.674001 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.674157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.674235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.674389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.674465 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.674624 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.675199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.675405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.675516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.675652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.675775 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.675889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676020 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.676847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677086 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.677985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678295 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678787 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.678934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679179 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679256 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.679987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680460 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680878 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.680956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.681908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.682866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.683929 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684007 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684416 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.684911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685106 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685348 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685427 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.685944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686559 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686793 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.686922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687844 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.687969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.688953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.689920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.690061 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.690182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.690307 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.690949 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.691914 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692041 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692167 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692293 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692418 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692542 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692803 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.692925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693599 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693669 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693877 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.693954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694539 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.694980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695498 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.695982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696831 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.696909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.697069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.697147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.697303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.697389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.697461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 16 19:03:51.697552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.697792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.698040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.698275 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.698757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 19:03:51.698928 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 16 19:03:51.726554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.726753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.726851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.726987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.727068 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.727210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.735495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.735807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.764551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.764779 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.764863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.765021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.818936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.819260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.867928 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.868162 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.868287 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:51.868520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.023948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024617 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024833 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.024914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.025052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.025143 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.025262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.048578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.048817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.048907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049150 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049280 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049498 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049711 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.049995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.050082 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.050183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.061244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.061387 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.074514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.074751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.074847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.074982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075206 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075557 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.075933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.076067 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 19:03:52.093237 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.093450 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 19:03:52.093534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.093692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 19:03:52.093799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.094001 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 19:03:52.094087 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.094210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 19:03:52.094290 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.094483 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.094574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.094745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.094826 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.095002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.095084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.095257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.104603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.104757 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.104891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.105038 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.105170 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.105309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.105460 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.105592 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.112681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.112773 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.115601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.115725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.115895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116182 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.116872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117656 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117850 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.117932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.118126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.118207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.118398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.118478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.141175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.141384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.142615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.142851 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.142933 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.143141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.148105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.148369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.148451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.148659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.148752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.148945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.154980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.155260 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.155361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.155578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.160903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.161201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.161283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.161488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.161569 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.161773 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.161855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162590 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.162939 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.163132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.178652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.178931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.179013 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.179221 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.182747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.183010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.185956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.186237 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.186319 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.186539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.190506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.190741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.193634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.193847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.196830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197169 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197574 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.197967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.198173 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.198255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.198454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.198543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.214565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.214655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.214857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.214938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.215124 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.215218 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.215413 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.215495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.222287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.222379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.223866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.224007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.225391 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.225473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.225669 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.225751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.228337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.228422 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.228631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.228715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.228917 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.229009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.229200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.229282 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.229587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.229671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.241401 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.241493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.241686 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.241768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.241968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.242062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.242252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.242334 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.242633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.242715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.242926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.243009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.243206 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.243308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.258538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.258706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.261444 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.261625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.264323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.264562 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.265759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.266007 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.267200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.267501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.267586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.267791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.267921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.268196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.274666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.275078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.276132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.276580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.278960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.279373 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.279523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.279662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.279923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.280018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.280215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.280498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.287792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.288083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.288362 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.288452 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.288747 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.289034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.295140 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.295444 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.295723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.295943 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 19:03:52.296026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.296233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.296315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.296520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.296603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.296799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.296879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297346 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297704 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.297975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.298200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.298287 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.298469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.298586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.298762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.298902 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.299032 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.299446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.300339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 19:03:52.333440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557b66fb99c0 pid: 81399 (default app) Sep 16 19:03:52.334799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557b66fb99c0 pid: 81400 (default app) Sep 16 19:03:52.336628 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15169c17-d18b-4f00-a119-28d6bcf00b79 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:03:52.336915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15169c17-d18b-4f00-a119-28d6bcf00b79 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:03:52.337188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15169c17-d18b-4f00-a119-28d6bcf00b79 None None] PATH_INFO: `/v3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:03:52.341319 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Fri Sep 16 19:03:52 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:03:59.093308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:03:59.093743 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:03:59.093845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:03:59.098771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:03:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:03:59.104339 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:03:59.104864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:03:59.104864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:03:59.174908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: INFO dbcounter [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] Registered counter for database keystone Sep 16 19:03:59.193717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG oslo_db.sqlalchemy.engines [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 16 19:03:59.199782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writer thread running {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 16 19:03:59.381213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 16 19:03:59.381513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 16 19:03:59.438737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:03:59.440344 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:03:59.564799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:03:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 16 19:04:00.029304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:00.098173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: INFO dbcounter [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] Registered counter for database keystone Sep 16 19:04:00.119262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG oslo_db.sqlalchemy.engines [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 16 19:04:00.125531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writer thread running {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 16 19:04:00.244532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f13c1-eccf-4693-b920-ff27fd97efe3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:00.248721 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:00.249337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:00.249617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:00.249884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:00.258454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:00.310351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 16 19:04:00.310629 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd2e6f-87c0-450b-af0e-dca5aa3eadfb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:00.320945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Fri Sep 16 19:04:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 19:04:00.326918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:00.344472 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac263-d953-4ae4-a734-87cbf3c447d6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:00.351841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:00.354061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:00.354361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:00.354658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] PATH_INFO: `/v3/projects/3a6c3a4a3b2c47b590674deda9305a68` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:00.356489 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] RBAC: Authorizing `identity:get_project(project_id=3a6c3a4a3b2c47b590674deda9305a68)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:00.410018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 16 19:04:00.410358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26864f31-4737-41ef-97f7-0c60e598aa23 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:00.414849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Fri Sep 16 19:04:00 2022] GET /identity/v3/projects/3a6c3a4a3b2c47b590674deda9305a68 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:02.528461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec4db-992d-4b05-ba7f-311f8d61dfd9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:02.528986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec4db-992d-4b05-ba7f-311f8d61dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:02.529078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec4db-992d-4b05-ba7f-311f8d61dfd9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:02.533113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:02.538485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:02.538906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:02.539032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:02.558502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:02.560009 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:02.568191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 16 19:04:02.833105 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec4db-992d-4b05-ba7f-311f8d61dfd9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:02.844821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec4db-992d-4b05-ba7f-311f8d61dfd9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:02.848982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238ed34-5ea5-465a-ac0b-563dcca03c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:02.849570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238ed34-5ea5-465a-ac0b-563dcca03c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:02.849849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238ed34-5ea5-465a-ac0b-563dcca03c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:02.850121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238ed34-5ea5-465a-ac0b-563dcca03c92 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:02.857528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238ed34-5ea5-465a-ac0b-563dcca03c92 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:02.859550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238ed34-5ea5-465a-ac0b-563dcca03c92 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:02.868530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Fri Sep 16 19:04:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 19:04:02.873581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:02.883164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1e69c-13ef-4c4b-857e-c56f021ea511 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:02.887510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0c267-b1e0-43f7-890b-b4ef10a64dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:02.888130 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0c267-b1e0-43f7-890b-b4ef10a64dee admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:02.888410 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0c267-b1e0-43f7-890b-b4ef10a64dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:02.888698 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0c267-b1e0-43f7-890b-b4ef10a64dee admin admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:02.889981 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0c267-b1e0-43f7-890b-b4ef10a64dee admin admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:02.891831 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0c267-b1e0-43f7-890b-b4ef10a64dee admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:02.893322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Fri Sep 16 19:04:02 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:05.085168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18632b-139a-48ba-b3bc-2cdabb95a486 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.085568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18632b-139a-48ba-b3bc-2cdabb95a486 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.085722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18632b-139a-48ba-b3bc-2cdabb95a486 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.087047 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:05.092154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.092561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.092710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.111199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:05.112481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:05.142491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e15e7-eeb7-4af2-a87d-fc122a91d146 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.142834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e15e7-eeb7-4af2-a87d-fc122a91d146 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.143118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e15e7-eeb7-4af2-a87d-fc122a91d146 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.144978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:05.150026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605d014-0e08-463a-9959-5f472e95380b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.150305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605d014-0e08-463a-9959-5f472e95380b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.150574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605d014-0e08-463a-9959-5f472e95380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.191983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-7605d014-0e08-463a-9959-5f472e95380b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 16 19:04:05.192371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-7605d014-0e08-463a-9959-5f472e95380b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 16 19:04:05.195725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:05.241396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7605d014-0e08-463a-9959-5f472e95380b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:05.242986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7605d014-0e08-463a-9959-5f472e95380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:05.339175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:05.462960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:05.490614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:05.569511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1605ac2-c8e4-424b-b7f2-e1fe68a49f0f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:05.602786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:05.603380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.603662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.603936 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.615447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605d014-0e08-463a-9959-5f472e95380b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:05.616120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:05.624964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:05.626053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:05.627021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:05.632095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:05.715908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605d014-0e08-463a-9959-5f472e95380b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:05.716094 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:05.718328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:05.735573 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4abc96-d489-4249-8df2-345d61910bdc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:05.755015 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:05.755968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.756246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.756514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.768299 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:05.768906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.769711 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.769983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.770640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:05.770937 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:05.772357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:05.783918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:05.789342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:05.823567 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:05.825081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:05.887800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da1d9f-1a74-4775-a2d2-2a444538fbf6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:05.924378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:05.925006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:05.925811 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:05.926085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:05.926520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 19:04:05.928010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:05.929458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:05.931146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:05 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:05.932117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:05.935831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:05.936119 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:05.951449 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:06.031802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942884c3-6114-4602-877c-6b97cf67e7d3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:06.034038 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61561c6-8b7f-41b7-bffe-0fb11e9f8a28 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:06.065888 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:06.066807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:06.067082 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:06.067376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:06.069191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:06.069973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:06.070259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:06.070534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:06.071722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:06.073164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:06.074690 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:06.080429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:06.086028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:06.087513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:04:06.105883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:06.106855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:04:06.107948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:04:06.107948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:04:06.107948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:06.107948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:06.113059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:06.160768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:06.170285 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:06.185782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd04c9c-6669-4d79-b3ec-2c1fdcb51224 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:06.218831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ff487-86c4-432c-a37c-4d4eab0caf9e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:06.222557 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:06.223169 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:06.223482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:06.223762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:06.224488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.domain.id=default, user.id=153cf9ce284c4a19b0247d5e1bc68384) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:06.224798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:06.226239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f389212-d79e-4cbf-a2ca-f32dc2182c1b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:06.233110 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Fri Sep 16 19:04:06 2022] GET /identity/v3/role_assignments?user.id=153cf9ce284c4a19b0247d5e1bc68384&scope.domain.id=default&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:06.255007 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:06.255671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:06.255948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:06.256217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:06.256875 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:06.257175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:06.258605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62561e80-fdd7-4a37-b1c5-73a9dc9ee861 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:06.278054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:08.620403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e448-c0c2-4041-92ad-8317b65f79ba None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:08.620907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e448-c0c2-4041-92ad-8317b65f79ba None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:08.621003 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e448-c0c2-4041-92ad-8317b65f79ba None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:08.622289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:08.627539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b15c2-fe23-4360-aa5f-1b38671fabf8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:08.628299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b15c2-fe23-4360-aa5f-1b38671fabf8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:08.629262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b15c2-fe23-4360-aa5f-1b38671fabf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:08.652275 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d66b15c2-fe23-4360-aa5f-1b38671fabf8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:08.653712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d66b15c2-fe23-4360-aa5f-1b38671fabf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:08.740681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:09.073284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfb282-5579-4bbf-ae66-849fc6a56cc1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.073665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfb282-5579-4bbf-ae66-849fc6a56cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.073851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfb282-5579-4bbf-ae66-849fc6a56cc1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.075376 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:09.080548 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.080838 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.081116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.108367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:09.109828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:09.160298 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e511-7a10-4676-8720-1509216fc96a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.160957 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e511-7a10-4676-8720-1509216fc96a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.161495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e511-7a10-4676-8720-1509216fc96a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.163107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 14/25] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:09.168871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc248cc1-3b43-477a-bd17-470862a1ea1a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.169468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:09.170629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc248cc1-3b43-477a-bd17-470862a1ea1a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.171134 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc248cc1-3b43-477a-bd17-470862a1ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.194170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bc248cc1-3b43-477a-bd17-470862a1ea1a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:09.196363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc248cc1-3b43-477a-bd17-470862a1ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:09.209240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 12/26] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:09.210405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:09.288899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 15/27] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:09.384585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286a190-6f0b-40c5-afa7-3696b7cabf90 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:09.392394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eb082-f8b8-4d04-93c4-6c300ca6478c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.392916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eb082-f8b8-4d04-93c4-6c300ca6478c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.393379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eb082-f8b8-4d04-93c4-6c300ca6478c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.400141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 16/28] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:09.409049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.409598 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.410582 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.441980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:09.443747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:09.461501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:09.462386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.462805 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.463217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.495871 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:09.502601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:09.503660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:09.504565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:09.506835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 13/29] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:09.511897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:09.537771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 17/30] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:09.538934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:09.542280 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:09.596278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:09.718131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264710bb-bd6d-4a96-909b-314103c0b779 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:09.740832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c98ab5-1ca9-4f98-983f-aec7798f0142 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:09.781131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:09.781759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.782040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.782316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.782990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:09.783309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:09.785220 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:09.801848 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 18/31] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:09.802673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:09.804993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:09.805630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:09.805911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:09.806181 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:09.827123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:09.828640 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:04:09.853760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:09.854628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:04:09.855434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:04:09.855434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:04:09.855434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:09.855434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 14/32] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:09.856113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:09.912613 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:09.961464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d416a4-b51e-4b62-9831-aab6ed343229 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:09.962523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:09.976323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd71db-1e20-437a-94de-2c222ddd76e8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:10.011166 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.012249 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.012671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.013086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.013228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.013841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.014240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.014650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.015059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.015163 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.016760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.018374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.020248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 15/33] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:09 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:10.021187 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:10.147136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a167fff-45fc-44df-8b50-a73e1b32f191 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:10.170289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 19/34] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Sep 16 19:04:09 2022] POST /identity/v3/roles => generated 219 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:10.171596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:10.202571 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.205436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.205980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.206423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.207919 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:10.208564 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.211069 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.233966 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 16/35] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:10.240785 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:10.279100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:10.393604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd614a34-a1e3-4a7a-b687-d4424b2aa3de None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:10.428281 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:10.439333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.439333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.439787 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.440076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.441175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.443019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.443764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7da07-634b-44b1-8acd-a2bd54563467 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:10.465147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 20/36] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Sep 16 19:04:10 2022] POST /identity/v3/roles => generated 225 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:10.465885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:10.489673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25daa6-0bd6-4461-bd4d-7810ad861d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.490592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25daa6-0bd6-4461-bd4d-7810ad861d7d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.491023 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25daa6-0bd6-4461-bd4d-7810ad861d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.491505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25daa6-0bd6-4461-bd4d-7810ad861d7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.493233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25daa6-0bd6-4461-bd4d-7810ad861d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.495590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25daa6-0bd6-4461-bd4d-7810ad861d7d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.498421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 17/37] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:10.500015 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324b42d-3e67-4ed0-96f6-e1105c7789a3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.500512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324b42d-3e67-4ed0-96f6-e1105c7789a3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.500979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324b42d-3e67-4ed0-96f6-e1105c7789a3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.502918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 18/38] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:10.505527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324b42d-3e67-4ed0-96f6-e1105c7789a3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:10.630558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2869a4d-a69f-482c-98dd-580ae648e0b2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:10.631750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:10.653862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324b42d-3e67-4ed0-96f6-e1105c7789a3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:10.688568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a8eb7-e320-48f7-90c8-4c027c57df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.689374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a8eb7-e320-48f7-90c8-4c027c57df7c None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.689662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a8eb7-e320-48f7-90c8-4c027c57df7c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.689947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a8eb7-e320-48f7-90c8-4c027c57df7c None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.690856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a8eb7-e320-48f7-90c8-4c027c57df7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.693807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a8eb7-e320-48f7-90c8-4c027c57df7c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.702870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037bde8-df3c-447c-a58f-60c4f9b94b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:10.703666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037bde8-df3c-447c-a58f-60c4f9b94b72 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.703942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037bde8-df3c-447c-a58f-60c4f9b94b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.704677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037bde8-df3c-447c-a58f-60c4f9b94b72 None admin] PATH_INFO: `/v3/domains/default/users/153cf9ce284c4a19b0247d5e1bc68384/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.706802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037bde8-df3c-447c-a58f-60c4f9b94b72 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=153cf9ce284c4a19b0247d5e1bc68384, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:10.708367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037bde8-df3c-447c-a58f-60c4f9b94b72 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:10.740117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 21/39] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:04:10 2022] POST /identity/v3/projects => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:10.741416 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.741710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.742040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.762576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:10.764151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:10.804020 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:10.835563 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 19/40] 10.222.0.18 () {58 vars in 1361 bytes} [Fri Sep 16 19:04:10 2022] PUT /identity/v3/domains/default/users/153cf9ce284c4a19b0247d5e1bc68384/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:04:10.841309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5290c1-a037-4997-af9a-9a4fff6ca5e5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.841981 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5290c1-a037-4997-af9a-9a4fff6ca5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.842492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5290c1-a037-4997-af9a-9a4fff6ca5e5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.849593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 20/41] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:10.856300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:10.856961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:10.857614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:10.861113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:10.886098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:10.887769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:10.969940 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:11.149363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:11.242383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:11.250722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b558e4d-a79f-400d-909f-54d94346f3c5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:11.286510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098f712-f120-491c-8772-35db48392e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:11.287135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098f712-f120-491c-8772-35db48392e22 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:11.287452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098f712-f120-491c-8772-35db48392e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:11.287736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098f712-f120-491c-8772-35db48392e22 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:11.288425 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098f712-f120-491c-8772-35db48392e22 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:11.289828 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098f712-f120-491c-8772-35db48392e22 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:11.306098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 23/44] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Sep 16 19:04:11 2022] POST /identity/v3/roles => generated 223 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:11.327222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:11.358316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc7a81-2d42-4127-9d17-7cc09e6264ca None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:11.397976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9f1f7-9267-4e26-9f4d-32dbc526989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:11.398640 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9f1f7-9267-4e26-9f4d-32dbc526989a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:11.398924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9f1f7-9267-4e26-9f4d-32dbc526989a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:11.399204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9f1f7-9267-4e26-9f4d-32dbc526989a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:11.400484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9f1f7-9267-4e26-9f4d-32dbc526989a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:11.402111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9f1f7-9267-4e26-9f4d-32dbc526989a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:11.403953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 22/45] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:11.409547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098f712-f120-491c-8772-35db48392e22 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:11.443430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:11.627090 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098f712-f120-491c-8772-35db48392e22 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:11.661192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d017a-43c1-4187-b0e3-7fe55d130f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:11.661807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d017a-43c1-4187-b0e3-7fe55d130f7b None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:11.662084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d017a-43c1-4187-b0e3-7fe55d130f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:11.662357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d017a-43c1-4187-b0e3-7fe55d130f7b None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:11.663014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d017a-43c1-4187-b0e3-7fe55d130f7b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:11.664494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d017a-43c1-4187-b0e3-7fe55d130f7b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:11.691774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:04:11 2022] POST /identity/v3/projects => generated 312 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:13.031885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e4ae8-3db1-4d20-b708-e00d2e55c877 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.032394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e4ae8-3db1-4d20-b708-e00d2e55c877 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.032504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e4ae8-3db1-4d20-b708-e00d2e55c877 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.033782 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:13.039599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97969e00-9918-4813-953b-09f882571975 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.039977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97969e00-9918-4813-953b-09f882571975 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.040157 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97969e00-9918-4813-953b-09f882571975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.058205 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-97969e00-9918-4813-953b-09f882571975 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:13.059461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-97969e00-9918-4813-953b-09f882571975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:13.124513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:13.388156 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e4ae8-3db1-4d20-b708-e00d2e55c877 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:13.487823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e4ae8-3db1-4d20-b708-e00d2e55c877 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:13.520856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:13.521480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.521756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.522025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.531728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:13.536637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:13.537443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:13.538201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 24/49] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:13.543672 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97969e00-9918-4813-953b-09f882571975 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:13.555137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:13.641055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97969e00-9918-4813-953b-09f882571975 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:13.674911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:13.676189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.676477 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.676755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.681754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:13.684853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:13.686260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce16d67-fe17-4216-9155-f0adffdde101 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:13.696365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 26/50] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:13.703962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:13.707118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:13.807631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c47e8-b7c3-4538-b631-72acbea4ac92 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:13.815932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031936aa-132c-4591-810f-e5b576f03262 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.816318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031936aa-132c-4591-810f-e5b576f03262 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.816673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031936aa-132c-4591-810f-e5b576f03262 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.818098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 27/51] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:13.823120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.823447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.823735 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.840498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:13.841117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.841210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:13.841402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.841675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.842335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:13.842781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:13.844242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:13.845810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:13 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:13.852021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:13.909015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 28/53] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:13.919024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:13.948440 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9351fa-5a73-457a-8afe-91a0711954dc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:13.980890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:13.981511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:13.981792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:13.982064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:13.983163 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:13.984907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:13.986436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:13.992247 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:13.994754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:14.090295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355ae1c-c0a1-4047-82f7-5f27accd7783 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:14.122926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:14.123557 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:14.123837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:14.124109 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:14.124823 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.domain.id=default, user.id=153cf9ce284c4a19b0247d5e1bc68384) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:14.125129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:14.126569 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:14.132151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1322 bytes} [Fri Sep 16 19:04:13 2022] GET /identity/v3/role_assignments?user.id=153cf9ce284c4a19b0247d5e1bc68384&scope.domain.id=default&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 530 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:14.179772 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:14.284208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b48695-d405-45fa-a97a-59f53f7d7785 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:14.426573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329efa6b-cb3f-4bba-915b-037e219f27ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:14.427191 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329efa6b-cb3f-4bba-915b-037e219f27ab None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:14.427495 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329efa6b-cb3f-4bba-915b-037e219f27ab None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:14.427767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329efa6b-cb3f-4bba-915b-037e219f27ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:14.428842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329efa6b-cb3f-4bba-915b-037e219f27ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:14.430302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329efa6b-cb3f-4bba-915b-037e219f27ab None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:14.431833 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:14.437576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:14.500609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:14.536110 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d441a-6742-4474-bc25-618b9f85037d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:14.568369 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91396738-f2d8-4ef7-a9fa-8032c6665412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:14.568968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91396738-f2d8-4ef7-a9fa-8032c6665412 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:14.569253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91396738-f2d8-4ef7-a9fa-8032c6665412 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:14.569526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91396738-f2d8-4ef7-a9fa-8032c6665412 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:14.570164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91396738-f2d8-4ef7-a9fa-8032c6665412 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:14.571630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91396738-f2d8-4ef7-a9fa-8032c6665412 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:14.597210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Fri Sep 16 19:04:14 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:16.746854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f40f-612b-4a9b-b267-1793dc87e033 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:16.747391 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f40f-612b-4a9b-b267-1793dc87e033 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:16.747478 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f40f-612b-4a9b-b267-1793dc87e033 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:16.748766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:16.754396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:16.754832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:16.754952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:16.776186 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:16.777366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:16.840994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:17.132551 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3f40f-612b-4a9b-b267-1793dc87e033 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:17.141268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:17.229914 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3f40f-612b-4a9b-b267-1793dc87e033 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:17.261045 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044535c-62e6-4d80-82d9-84ba80441766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:17.261658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044535c-62e6-4d80-82d9-84ba80441766 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:17.261934 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044535c-62e6-4d80-82d9-84ba80441766 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:17.262202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044535c-62e6-4d80-82d9-84ba80441766 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:17.263283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044535c-62e6-4d80-82d9-84ba80441766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:17.264731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044535c-62e6-4d80-82d9-84ba80441766 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:17.266246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:17.272186 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:17.318080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:17.366828 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07089639-5a7c-4ab0-b089-b677c341196c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:17.404922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccecc46-7ec1-4202-8e0e-16460c24c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:17.405726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccecc46-7ec1-4202-8e0e-16460c24c24b None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:17.406006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccecc46-7ec1-4202-8e0e-16460c24c24b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:17.406278 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccecc46-7ec1-4202-8e0e-16460c24c24b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:17.406928 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccecc46-7ec1-4202-8e0e-16460c24c24b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:17.408598 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccecc46-7ec1-4202-8e0e-16460c24c24b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:17.435435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:17 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:20.586006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a27c-a893-4e5c-ba1b-11b973a5afdc None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:20.586519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a27c-a893-4e5c-ba1b-11b973a5afdc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:20.586589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a27c-a893-4e5c-ba1b-11b973a5afdc None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:20.588053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:20.593116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7343ec-3043-4299-b06d-171dc2952458 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:20.593498 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7343ec-3043-4299-b06d-171dc2952458 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:20.593657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7343ec-3043-4299-b06d-171dc2952458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:20.615514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9b7343ec-3043-4299-b06d-171dc2952458 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:20.616693 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b7343ec-3043-4299-b06d-171dc2952458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:20.712886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 33/63] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:20.973848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83ae17-799c-448d-8bd2-6038d0c9e5f3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:20.974188 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83ae17-799c-448d-8bd2-6038d0c9e5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:20.974356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83ae17-799c-448d-8bd2-6038d0c9e5f3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:20.975965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 31/64] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:20.989152 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449ebe-19a8-4164-8434-c5a76ef9c87a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:20.989458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449ebe-19a8-4164-8434-c5a76ef9c87a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:20.989706 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449ebe-19a8-4164-8434-c5a76ef9c87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.011954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4a449ebe-19a8-4164-8434-c5a76ef9c87a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:21.013135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4a449ebe-19a8-4164-8434-c5a76ef9c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:21.052763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.103074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 34/65] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:21.112572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36880132-86c1-4edf-8985-b673cd0fa192 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.112865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36880132-86c1-4edf-8985-b673cd0fa192 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.113130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36880132-86c1-4edf-8985-b673cd0fa192 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.114607 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 32/66] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:21.120525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61590833-b6c6-4f88-ac1d-b00b181ea98e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.120815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61590833-b6c6-4f88-ac1d-b00b181ea98e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.121096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61590833-b6c6-4f88-ac1d-b00b181ea98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.144875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-61590833-b6c6-4f88-ac1d-b00b181ea98e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:21.146086 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-61590833-b6c6-4f88-ac1d-b00b181ea98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:21.206126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36880132-86c1-4edf-8985-b673cd0fa192 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.266275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 35/67] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:21.268028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124a86-312a-4126-ab7c-fd64932f22c4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.268604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124a86-312a-4126-ab7c-fd64932f22c4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.269081 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124a86-312a-4126-ab7c-fd64932f22c4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.270835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 36/68] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:21.276399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e0df1-37fe-445e-a9df-89aafa9a3188 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.276930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e0df1-37fe-445e-a9df-89aafa9a3188 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.277417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e0df1-37fe-445e-a9df-89aafa9a3188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.305311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a00e0df1-37fe-445e-a9df-89aafa9a3188 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:21.306392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.307801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a00e0df1-37fe-445e-a9df-89aafa9a3188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:21.336541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36880132-86c1-4edf-8985-b673cd0fa192 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:21.379942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:21.380847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.381399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.381820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.395653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:21.402171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:21.403274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:21.404240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:21.404929 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 37/69] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:21.405500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e4b0-8506-4c83-890c-5eb5f29f1bd2 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.405851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e4b0-8506-4c83-890c-5eb5f29f1bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.406192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e4b0-8506-4c83-890c-5eb5f29f1bd2 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.406661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 33/70] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:21.407591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 38/71] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:21.412218 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.416344 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.416858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.417329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.439384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:21.440924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:21.463639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.520671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 39/72] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:21.527309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.561414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340758e-6d4e-4a1c-a825-b4b650fc092a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:21.595046 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:21.596401 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.596823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.597233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:21.598590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:21.600151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:21.601885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.613819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 34/73] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:21.614931 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.682440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213affa-5a2b-46ae-a444-73859ee0d953 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:21.718871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:21.719250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5aad1-2de2-49fa-8c44-f10d5c5c18c4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:21.719547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.719838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.720118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.733775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:21.739022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:21.739885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:21.740636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 40/74] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:21.741307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.757742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.760695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:21.761336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.761611 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.761892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.772960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:21.777693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:21.778514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:21.779731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 35/75] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:21.781139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.838700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.852308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b1d87-5920-41f1-a9f5-d88444b4d962 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:21.883862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a24e5d-cf0a-4bb4-a939-2f1028e8e1ac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:21.885499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:21.886127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.886405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.886677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.887813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:21.889268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:21.890987 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 41/76] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:21.891606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.919350 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:21.919975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:21.920252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:21.920519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:21.921625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:21.923057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:21.924627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 36/77] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:21.925378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:21.972091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:21.991460 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5830c-0e95-421c-bff7-27fb1ac86245 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.022340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639526c-4b6d-494d-8486-6033c5ab5e3f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.025308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.025859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.026145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.026423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.027089 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:22.027440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.028841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.037363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 42/78] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:22.038156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.046872 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.058532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.059140 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.059459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.059739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.060525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:22.060824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.062297 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.071397 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 37/79] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:04:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:22.072214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.140318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa7ac3-7e24-4739-bbae-0111eeac4aef None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.169220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2d103-19f9-4590-9daf-0862ff1d370d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.173850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.174480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.174758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.175030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.185350 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:22.190653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:22.192138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:22.192909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 43/80] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:22.193526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.196676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.207836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.208454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.208738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.209025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.261743 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.276312 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.277709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.290029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222a8a9-7101-41f1-84b6-e5f08883fc7a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.322362 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.322972 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.323254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.323564 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.324216 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.325654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.346432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 44/81] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:22 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:22.348889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.382570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 38/82] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:22.388766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.415147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.466788 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0837-21c2-4555-92b8-93cb684eae45 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.490028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.505270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.505927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.506210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.506483 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.508226 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024e4f3-1351-42d9-a142-286915e3a494 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.524087 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.525977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.527743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 45/83] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:22.528464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.548164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.548803 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.549084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.549357 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.550492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.552009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.553594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 39/84] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:22.554356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.649028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be51f2-61a6-48ec-9a16-094569963c77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.668712 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.682974 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b190-2d92-42f7-a95f-89e86d5d59fd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.693492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.694191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.694463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.694793 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.695532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:22.695838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.697235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.710531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 46/85] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:22.711464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.728394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.729055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.729337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.729617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.730776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.732299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.734108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 40/86] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:22.734981 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:22.760273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.843533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5335a-b84e-4a15-9eb8-d61406e14b21 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.853084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52e07d-b350-454a-9fd7-c84b3412047f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:22.884451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.885670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.886131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.886832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.899147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:22.899803 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:22.900085 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:22.900415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:22.953060 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:22.974778 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.976175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:22.984878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:22.986537 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:23.013858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:23.100708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 47/87] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:23.104227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.109488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 41/88] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:23.110816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.215930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:23.218151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9f2a3-e819-4e49-b9ab-81f57c6c6104 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:23.242922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5351f4-917c-4822-88c9-61a208ff34ad None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:23.244977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:23.258063 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:23.259107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:23.259445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:23.259728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:23.260467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=153cf9ce284c4a19b0247d5e1bc68384) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:23.260788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:23.262215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:23.273405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 42/89] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/role_assignments?user.id=153cf9ce284c4a19b0247d5e1bc68384&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:23.274274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.290427 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:23.291775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:23.292769 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:23.293374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:23.295162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:23.297243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:23.299761 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 48/90] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:23.304470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.463078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d0912-3f28-4f7a-aa72-4345dd104f77 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:23.512466 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:23.513112 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:23.513628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:23.513918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:23.515048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:23.516612 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:23.518474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 43/91] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:23.519147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.529291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bb481-b595-4746-baa4-fd194391dcaa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:23.581652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:23.629608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:23.630314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:23.630603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:23.630882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:23.632932 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a83a2-4998-4965-8de2-fb12bbc5a167 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:23.635988 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:23.637452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:23.639458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 49/92] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:23.640241 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.670797 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:23.671435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:23.671769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:23.672046 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:23.694116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:23.753239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:23.755824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:23.817033 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:23.870677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 44/93] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:23.871571 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:23.925494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49ee9e-6e4b-4ab6-b65e-d481803432ca None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:23.981532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36262f-88ce-42c3-9d83-7fe0cc537315 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:24.019228 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:24.020328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:24.020607 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:24.020881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:24.021603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] RBAC: Adding query filter params (role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=ff702c07bb134279b453ea6ea09d38b2) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:24.021911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:24.023345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:24.029216 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 45/94] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/role_assignments?user.id=ff702c07bb134279b453ea6ea09d38b2&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:24.030071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:24.037054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:24.037683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:24.037965 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:24.038240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:24.038976 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=ff702c07bb134279b453ea6ea09d38b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:24.039306 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:24.040754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:24.050735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:24.065420 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 50/95] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:23 2022] GET /identity/v3/role_assignments?user.id=ff702c07bb134279b453ea6ea09d38b2&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=f2e63848ed864ae5901749a96467215f => generated 259 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:24.149462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400baa-7bef-46b7-83f1-f044f3841eca None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:24.190124 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59656cd0-c76a-48de-941f-105d18a116aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:24.190789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59656cd0-c76a-48de-941f-105d18a116aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:24.191068 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59656cd0-c76a-48de-941f-105d18a116aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:24.191907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59656cd0-c76a-48de-941f-105d18a116aa None admin] PATH_INFO: `/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:24.193226 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59656cd0-c76a-48de-941f-105d18a116aa None admin] RBAC: Authorizing `identity:get_project(project_id=e216823ad4974e0eb3ad961d3bbc0b1c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:24.194850 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59656cd0-c76a-48de-941f-105d18a116aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:24.196709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 46/96] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:24 2022] GET /identity/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:24.203033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:24.340633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:24.366238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c4fc-c2b2-4cfe-b530-eca8ffcf2272 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:24.417478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:24.418112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:24.418397 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:24.418678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:24.419454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=e216823ad4974e0eb3ad961d3bbc0b1c, user.id=ff702c07bb134279b453ea6ea09d38b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:24.419772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:24.421270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c55d9-4b7e-4730-a448-62791ac21795 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:24.432248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 51/97] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:24 2022] GET /identity/v3/role_assignments?user.id=ff702c07bb134279b453ea6ea09d38b2&scope.project.id=e216823ad4974e0eb3ad961d3bbc0b1c&role.id=f2e63848ed864ae5901749a96467215f => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:25.184255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c1cad-f683-47e3-b376-b38af0895ad2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:25.184748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c1cad-f683-47e3-b376-b38af0895ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:25.184847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c1cad-f683-47e3-b376-b38af0895ad2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:25.186434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 47/98] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:25.192015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d3060-22a5-43c5-8e67-fd9e5ae0774d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:25.192477 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d3060-22a5-43c5-8e67-fd9e5ae0774d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:25.192562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d3060-22a5-43c5-8e67-fd9e5ae0774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:25.211916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ce8d3060-22a5-43c5-8e67-fd9e5ae0774d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:25.213111 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ce8d3060-22a5-43c5-8e67-fd9e5ae0774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:25.283490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 52/99] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:25.307415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c6d3d-27cb-47c6-8812-ae3ed2fd5789 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:25.307723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c6d3d-27cb-47c6-8812-ae3ed2fd5789 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:25.308198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c6d3d-27cb-47c6-8812-ae3ed2fd5789 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:25.310609 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 48/100] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:25.317048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:25.317332 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:25.317611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:25.337231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:25.338329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:25.406769 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 53/101] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:25.643682 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c6d3d-27cb-47c6-8812-ae3ed2fd5789 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:25.757908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:25.778819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c6d3d-27cb-47c6-8812-ae3ed2fd5789 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:25.829941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:25.830580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:25.830859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:25.831181 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:25.843957 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:25.848203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:25.853196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:25.854150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:25.854895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 49/102] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:25.859168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:25.862183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:26.033400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f97db-a44a-46f4-b189-65633bc66b71 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:26.115903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694f40a-2e86-47c0-87f9-eeefe9ad2383 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:26.140917 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:26.141580 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.141862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.142140 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.147936 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:26.149403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:26.151265 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 54/103] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:26.156957 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:26.180566 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:26.181302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.181588 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.181864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.182536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:26.182843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:26.184368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:26.201233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 50/104] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:26.206538 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:26.227527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:26.290959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:26.337137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c753b5-21eb-4763-b12a-87bc9b335a4d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:26.413999 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:26.414640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.414923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.415201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.415901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:26.417425 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:26.442938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddf9a1-a0e8-4be8-a259-b1b7cb13fd2d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:26.475475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 55/105] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:04:26 2022] POST /identity/v3/projects => generated 302 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:26.507282 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc8ead-fce5-4665-b9ff-18904f18b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:26.507969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc8ead-fce5-4665-b9ff-18904f18b459 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.508252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc8ead-fce5-4665-b9ff-18904f18b459 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.508528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc8ead-fce5-4665-b9ff-18904f18b459 None admin] PATH_INFO: `/v3/users/9535d88e0c414427b19e4c50f73d786e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.534059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc8ead-fce5-4665-b9ff-18904f18b459 None admin] RBAC: Authorizing `identity:get_user(user_id=9535d88e0c414427b19e4c50f73d786e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:26.536164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc8ead-fce5-4665-b9ff-18904f18b459 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:26.537877 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 51/106] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:26 2022] GET /identity/v3/users/9535d88e0c414427b19e4c50f73d786e => generated 289 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:26.543930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:26.565987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e8055-d07c-4699-85e7-06c5245f49b4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.566292 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e8055-d07c-4699-85e7-06c5245f49b4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.567158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e8055-d07c-4699-85e7-06c5245f49b4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.568817 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 52/107] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:26.574167 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.574456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.574734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.611048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:26.612303 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:26.721009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:26.733979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 53/108] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:26.774154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644ce13-69ff-41dd-a460-02f9435c6389 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:26.804659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:26.821630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:26.822250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:26.822532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:26.822808 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:26.911600 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:26.913074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:27.029728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 56/109] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:27.035710 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:27.104233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:27.135026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:27.193259 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac18e5-751f-44e6-b2ef-9497c58785dd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:27.235579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d254ec6-9dad-43fa-8c53-d57bab47b2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:27.240576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d254ec6-9dad-43fa-8c53-d57bab47b2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.240873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d254ec6-9dad-43fa-8c53-d57bab47b2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.241158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d254ec6-9dad-43fa-8c53-d57bab47b2d9 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.244179 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d254ec6-9dad-43fa-8c53-d57bab47b2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:27.245638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d254ec6-9dad-43fa-8c53-d57bab47b2d9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:27.247299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 54/110] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:27 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:27.248442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46918b-1ed4-4d20-85e8-48de83d24526 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.248726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46918b-1ed4-4d20-85e8-48de83d24526 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.249002 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46918b-1ed4-4d20-85e8-48de83d24526 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.250201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 55/111] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:27.261979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46918b-1ed4-4d20-85e8-48de83d24526 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:27.326666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5513f-ac1a-4376-95cd-1e54f7978ccf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:27.385338 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:27.397330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cc489-4262-4c1f-80f1-c0b87aec4cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:27.397983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cc489-4262-4c1f-80f1-c0b87aec4cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.398264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cc489-4262-4c1f-80f1-c0b87aec4cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.398541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cc489-4262-4c1f-80f1-c0b87aec4cc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.420490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cc489-4262-4c1f-80f1-c0b87aec4cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-732cc489-4262-4c1f-80f1-c0b87aec4cc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:27.425614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:27.426739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:27.427732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:27.431256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 57/112] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:27.431256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19443f06-8bff-470e-ba55-758cbae74b3c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.431256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19443f06-8bff-470e-ba55-758cbae74b3c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.431256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19443f06-8bff-470e-ba55-758cbae74b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.448858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46918b-1ed4-4d20-85e8-48de83d24526 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:27.455207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-19443f06-8bff-470e-ba55-758cbae74b3c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:27.456533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-19443f06-8bff-470e-ba55-758cbae74b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:27.493952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:27.494585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.495141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.495453 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.496928 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] RBAC: Adding query filter params (role.id=7561f8727a4348488f94576d15c61021, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=9535d88e0c414427b19e4c50f73d786e) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:27.497266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:27.498763 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ece6a-ad71-40bf-b868-54f7a3089775 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:27.506845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 56/113] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:27 2022] GET /identity/v3/role_assignments?user.id=9535d88e0c414427b19e4c50f73d786e&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=7561f8727a4348488f94576d15c61021 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:27.508671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9448-8f5f-49b2-8e14-21ed10079cfe None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.508962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9448-8f5f-49b2-8e14-21ed10079cfe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.509239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad9448-8f5f-49b2-8e14-21ed10079cfe None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.511336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 57/114] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:27.512683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad9448-8f5f-49b2-8e14-21ed10079cfe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:27.558250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 58/115] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:27.563856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6efcb7-7ba0-4d8b-b67e-9efece5557dc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.564167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6efcb7-7ba0-4d8b-b67e-9efece5557dc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.564426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6efcb7-7ba0-4d8b-b67e-9efece5557dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.603077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ef6efcb7-7ba0-4d8b-b67e-9efece5557dc None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:27.604381 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ef6efcb7-7ba0-4d8b-b67e-9efece5557dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:27.622400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:27.716442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 59/116] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:27.717440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfb0b9-1816-4659-adf9-f8f5f812c4fe None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.717727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfb0b9-1816-4659-adf9-f8f5f812c4fe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.718005 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfb0b9-1816-4659-adf9-f8f5f812c4fe None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.719256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 60/117] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:27.725865 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.726134 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.726413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.732092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad9448-8f5f-49b2-8e14-21ed10079cfe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:27.774237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:27.775468 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:27.790303 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:27.802568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:27.803186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:27.803505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:27.803777 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:27.804447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:27.804747 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:27.806125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:27.825780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 58/118] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:27.833965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:27.897585 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 61/119] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:27.917487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:27.969091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22ea4d-48bf-409b-abab-19a778c1b260 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.015313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.015313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.015313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.015313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.015713 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:28.015713 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:28.016220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 59/120] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:27 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:28.022798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.098618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:28.125358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:28.166024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0df790-959e-49f3-b6bc-fc06a153dc78 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.185403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34f584-26f2-467d-a106-986da5566e75 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.205402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.206051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.206330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.206605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.209644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:28.211070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:28.212935 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 60/121] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:28.213475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.249935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.250576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.250857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.251133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.271867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:28.281911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:28.282772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:28.283930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:28.286021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 62/122] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:04:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:28.287206 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.359246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14580d-19ec-4ef3-8724-7e20b739d759 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.388197 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:28.399512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.400237 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.400340 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0fd6a-43cc-49dc-b967-b5389885b128 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.400732 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.400802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.416998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:28.422381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:28.423200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:28.424147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 61/123] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:28.424980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.437495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.438239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.438587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.438930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/153cf9ce284c4a19b0247d5e1bc68384/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.441102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac2ffc24a14d4dc1a43619feba9ae47b, user_id=153cf9ce284c4a19b0247d5e1bc68384, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:28.442684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:28.445096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:28.459707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 63/124] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:28 2022] PUT /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/153cf9ce284c4a19b0247d5e1bc68384/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:28.460768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.576455 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949bc3f-e8d6-498e-a750-a0e695aadc85 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.592297 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7a33b-4713-4ed3-9936-82e8c9d862cc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.624903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.625539 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.625821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.626098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.626792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:28.627101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:28.631127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:28.651954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 64/125] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:28.653784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.666201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.666850 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.667127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.667432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.689742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:28.699208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:28.701593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 62/126] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:28.703657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.761016 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:28.792348 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:28.811031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbb55c-fc25-4008-aca5-e4e5d267e144 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.846284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.847214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.848286 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.848755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.849676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:28.850864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:28.852626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:28.860152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cc0e1-daf9-4dcf-83ce-948716879ddc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:28.862224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 65/127] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:28.863357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:28.903618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:28.904252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:28.904535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:28.904814 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:28.905994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:28.908004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:28.909993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 63/128] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:28.911719 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.005799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9652e8-3dca-4413-befb-5ea721001d83 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.044989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:29.078322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.078950 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.079232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.081226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.081918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:29.082821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.084238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.090694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9036b65-7d56-4716-9a66-6d3877389216 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.106394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 66/129] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:29.107212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.119714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:29.160016 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.160841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.161122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.161396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.163474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.164918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.166720 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 64/130] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:28 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:29.167277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.259703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f9dc1-c2a1-48b7-8202-33f778b87dc9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.336096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.336764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.337048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.337326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.338560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.340103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.341780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 67/131] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:29 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:29.342565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.358765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f3d32-5a40-4577-8f4c-545baddeb6bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.396268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.396910 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.397194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.397470 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.398711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.400091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.401688 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 65/132] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:29 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:29.402496 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.461018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:29.463261 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:29.531395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e402-b285-4562-bbe1-90f96cdec9b1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.533284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7db85-ea89-40dd-a2e1-23c66dd84536 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.568301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147390ad-954e-4588-8994-6bef826e035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.568944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147390ad-954e-4588-8994-6bef826e035a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.569229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147390ad-954e-4588-8994-6bef826e035a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.569507 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147390ad-954e-4588-8994-6bef826e035a None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.570634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147390ad-954e-4588-8994-6bef826e035a None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.572249 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147390ad-954e-4588-8994-6bef826e035a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.574120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 68/133] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:29 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:29.574859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147390ad-954e-4588-8994-6bef826e035a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.575633 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995f8d3-9162-4f5b-94d6-1b9d3a6e94c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.576376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f8d3-9162-4f5b-94d6-1b9d3a6e94c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.576724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f8d3-9162-4f5b-94d6-1b9d3a6e94c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.577128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f8d3-9162-4f5b-94d6-1b9d3a6e94c0 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/ff702c07bb134279b453ea6ea09d38b2/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.580595 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f8d3-9162-4f5b-94d6-1b9d3a6e94c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac2ffc24a14d4dc1a43619feba9ae47b, user_id=ff702c07bb134279b453ea6ea09d38b2, role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.582024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f8d3-9162-4f5b-94d6-1b9d3a6e94c0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.600596 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 66/134] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:29 2022] PUT /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/ff702c07bb134279b453ea6ea09d38b2/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:29.604916 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc5b6-0846-4122-a6f5-50ef1a28288b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.605207 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc5b6-0846-4122-a6f5-50ef1a28288b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.605517 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc5b6-0846-4122-a6f5-50ef1a28288b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.607258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 67/135] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:29.614260 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc5b6-0846-4122-a6f5-50ef1a28288b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:29.752714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc5b6-0846-4122-a6f5-50ef1a28288b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.768823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:29.811923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7da6a-bbf6-4fb2-a14d-62ab2288a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.812625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7da6a-bbf6-4fb2-a14d-62ab2288a6b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.812915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7da6a-bbf6-4fb2-a14d-62ab2288a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.813200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7da6a-bbf6-4fb2-a14d-62ab2288a6b4 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/ff702c07bb134279b453ea6ea09d38b2/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.815685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7da6a-bbf6-4fb2-a14d-62ab2288a6b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac2ffc24a14d4dc1a43619feba9ae47b, user_id=ff702c07bb134279b453ea6ea09d38b2, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.817616 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7da6a-bbf6-4fb2-a14d-62ab2288a6b4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.836148 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 68/136] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:29 2022] PUT /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/ff702c07bb134279b453ea6ea09d38b2/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:29.837632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57748-c820-4ec5-991f-9e9a71c5016c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.838017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57748-c820-4ec5-991f-9e9a71c5016c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.838365 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57748-c820-4ec5-991f-9e9a71c5016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.870240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-59d57748-c820-4ec5-991f-9e9a71c5016c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:29.872339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-59d57748-c820-4ec5-991f-9e9a71c5016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:29.878082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147390ad-954e-4588-8994-6bef826e035a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:29.909320 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:29.945078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:29.945749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:29.946034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:29.946313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] PATH_INFO: `/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:29.952177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] RBAC: Authorizing `identity:get_project(project_id=e216823ad4974e0eb3ad961d3bbc0b1c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:29.953680 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:29.955920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 69/137] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:29 2022] GET /identity/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c => generated 312 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:29.965354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:30.012507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 69/138] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:30.108424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b18dff-0941-4d51-9058-24e4651183bd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:30.149850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116f5c6-aacb-4814-a120-7c51579090c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:30.150452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f5c6-aacb-4814-a120-7c51579090c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:30.150737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f5c6-aacb-4814-a120-7c51579090c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:30.151014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f5c6-aacb-4814-a120-7c51579090c3 None admin] PATH_INFO: `/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c/users/ff702c07bb134279b453ea6ea09d38b2/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:30.153051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116f5c6-aacb-4814-a120-7c51579090c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e216823ad4974e0eb3ad961d3bbc0b1c, user_id=ff702c07bb134279b453ea6ea09d38b2, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:30.154513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116f5c6-aacb-4814-a120-7c51579090c3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:30.179734 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 70/139] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:29 2022] PUT /identity/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c/users/ff702c07bb134279b453ea6ea09d38b2/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:04:30.237973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c025bab-a8ea-4e45-af3f-d069c47ed9c5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:30.238257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c025bab-a8ea-4e45-af3f-d069c47ed9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:30.239210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c025bab-a8ea-4e45-af3f-d069c47ed9c5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:30.240911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 70/140] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:30.247167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:30.247532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:30.247820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:30.268608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:30.269910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:30.327838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:30.372171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 71/141] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:30.492531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c025bab-a8ea-4e45-af3f-d069c47ed9c5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:30.619818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:30.624525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c025bab-a8ea-4e45-af3f-d069c47ed9c5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:30.669412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:30.670050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:30.670331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:30.670610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:30.676581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:30.678046 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:30.679967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 71/142] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:30.685482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:30.739066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:30.851268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16dcd5-7520-4927-8415-a5ecd4d31c5f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:30.865242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa561c-4e31-420c-b707-1143443ed5b7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:30.888873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:30.889620 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:30.889978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:30.890330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:30.901856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:30.907098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:30.907993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:30.909050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 72/143] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:30.915414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:30.930140 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:30.931036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:30.931514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:30.931990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:30.932835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:30.934462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:30.951671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:30.965480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 72/144] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:30 2022] POST /identity/v3/users => generated 283 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:31.043603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0473c1-0e8b-4a68-a6fb-57e7be8d558c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:31.096884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:31.098513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.099001 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.099568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.100463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:31.100969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:31.102652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc0c9f-04be-4d23-bb0e-8210e883dfd1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:31.112821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 73/145] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:31.124488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:31.139472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392b1a9-cc90-4716-9fe1-c14340a0bc9f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.140107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392b1a9-cc90-4716-9fe1-c14340a0bc9f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.140595 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392b1a9-cc90-4716-9fe1-c14340a0bc9f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.142917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 74/146] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:31.148823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.149115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.149395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.169484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:31.170708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:31.187548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:31.252669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 75/147] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:31.302017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656d63-68cb-4800-92c5-c83ff506cc62 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:31.348549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:31.349198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.349476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.349747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] PATH_INFO: `/v3/users/9535d88e0c414427b19e4c50f73d786e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.350923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=9535d88e0c414427b19e4c50f73d786e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:31.352403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:31.354182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 73/148] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:31 2022] GET /identity/v3/users/9535d88e0c414427b19e4c50f73d786e => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:31.360319 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:31.398901 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:31.515109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09e1af-1193-40a5-be55-15fbfe7a9403 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:31.564653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:31.568580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.569091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.569585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.571371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:31.573059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:31.576472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 76/149] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:31 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:31.580448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:31.582728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:31.701217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:31.719244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4d911-91a8-441e-9009-1e8669e0aede None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:31.719637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:31.722213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973ff83-391a-4be0-a690-1628b823f5b5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:31.766501 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:31.766988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:31.767194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.767497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.767774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.768529 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.769019 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.769513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/9535d88e0c414427b19e4c50f73d786e/roles/7561f8727a4348488f94576d15c61021` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.772761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] RBAC: Authorizing `identity:create_grant(project_id=ac2ffc24a14d4dc1a43619feba9ae47b, user_id=9535d88e0c414427b19e4c50f73d786e, role_id=7561f8727a4348488f94576d15c61021)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:31.774865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:31.781528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:31.788034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:31.789010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:31.790049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 74/150] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:31.799987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 77/151] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:31 2022] PUT /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/users/9535d88e0c414427b19e4c50f73d786e/roles/7561f8727a4348488f94576d15c61021 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:04:31.803563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:31.932703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673acf93-7c4e-4079-ba73-ccafaeb38c38 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:31.971700 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:31.972664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:31.973690 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:31.974153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:31.975034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:31.975570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:31.977215 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:31.987431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 75/152] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:31.993693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:32.102522 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080ac4c-23ed-43ee-9268-fd3d0ea464af None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:32.139083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:32.139762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:32.140044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:32.140321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] PATH_INFO: `/v3/users/153cf9ce284c4a19b0247d5e1bc68384` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:32.141463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] RBAC: Authorizing `identity:get_user(user_id=153cf9ce284c4a19b0247d5e1bc68384)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:32.142917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:32.145457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 78/153] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:31 2022] GET /identity/v3/users/153cf9ce284c4a19b0247d5e1bc68384 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:32.151013 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:32.174885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:32.264754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a1e7-8de6-4ad9-9b15-a59a9ca4a81d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:32.302420 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:32.303040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:32.303348 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:32.303629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:32.381608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:32.383180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:32.405160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:32.496715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 76/154] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:32.502304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:32.527129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:32.645953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7b75-fe7f-459a-b932-c7cbc1b54367 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:32.687782 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447285-d2db-4bca-8096-ec8a19300dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:32.688432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447285-d2db-4bca-8096-ec8a19300dcb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:32.688840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447285-d2db-4bca-8096-ec8a19300dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:32.689107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447285-d2db-4bca-8096-ec8a19300dcb None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:32.690364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d447285-d2db-4bca-8096-ec8a19300dcb None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:32.692410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d447285-d2db-4bca-8096-ec8a19300dcb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:32.696383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 79/155] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:32 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:32.700211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:32.820258 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00479a1-00c5-4485-82df-c916f006a476 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:32.844475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:32.859702 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:32.860583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:32.860866 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:32.861144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:32.861901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=153cf9ce284c4a19b0247d5e1bc68384) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:32.862529 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:32.865707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b91ef-8c5d-4602-b10e-7699cecbbed9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:32.873356 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 77/156] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:32 2022] GET /identity/v3/role_assignments?user.id=153cf9ce284c4a19b0247d5e1bc68384&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:33.434998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8561db-4c70-4c09-bf75-ed4192aa2994 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:33.435354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8561db-4c70-4c09-bf75-ed4192aa2994 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:33.435475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8561db-4c70-4c09-bf75-ed4192aa2994 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:33.437062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 80/157] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:33.442664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073667-5d73-4325-88e4-bc16ac4d379c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:33.442950 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073667-5d73-4325-88e4-bc16ac4d379c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:33.443227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073667-5d73-4325-88e4-bc16ac4d379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:33.463709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9b073667-5d73-4325-88e4-bc16ac4d379c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:33.464893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b073667-5d73-4325-88e4-bc16ac4d379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:33.538416 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 78/158] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:33.902441 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621391a-6b1d-46c8-bb56-5222808b23be None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:33.902970 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621391a-6b1d-46c8-bb56-5222808b23be None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:33.903057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621391a-6b1d-46c8-bb56-5222808b23be None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:33.904718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 81/159] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:33.919952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:33.920553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:33.920553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:33.947761 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:33.949011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:34.042799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 79/160] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:34.104547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621391a-6b1d-46c8-bb56-5222808b23be None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:34.216416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:34.238056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621391a-6b1d-46c8-bb56-5222808b23be None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:34.279973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:34.280603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:34.280879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:34.281150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:34.296118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:34.301540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:34.302457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:34.303234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 82/161] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:34.310034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:34.366505 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:34.447370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc015fc-7d9a-41ed-8cd6-b6c1e9ded30b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:34.499389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:34.500037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:34.500316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:34.500590 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:34.501282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:34.501586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:34.503011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:34.515993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 80/162] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:34.522147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:34.629684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:34.653674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848ab5-e4de-4f0b-adb4-20406cf08018 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:34.717973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd62dac-bf85-406a-a975-dbefdb922ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:34.718737 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62dac-bf85-406a-a975-dbefdb922ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:34.719017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62dac-bf85-406a-a975-dbefdb922ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:34.719317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62dac-bf85-406a-a975-dbefdb922ab8 None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:34.723971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd62dac-bf85-406a-a975-dbefdb922ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:34.726091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd62dac-bf85-406a-a975-dbefdb922ab8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:34.728352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 83/163] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:34 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:34.731349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7e682-b80b-4641-b284-0afdfef1c4a8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:34.731349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7e682-b80b-4641-b284-0afdfef1c4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:34.731349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7e682-b80b-4641-b284-0afdfef1c4a8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:34.731349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 84/164] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:34.737214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:34.737499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:34.737771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:34.775968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:34.777087 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:34.784096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19243efd-dbb1-4ff7-a233-fa4f79363724 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:34.799644 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:34.800212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:34.851232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:34.851905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:34.852184 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:34.852457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:34.877327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:34.883323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:34.884185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:34.885190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:34.889658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 81/165] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:34.889658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:34.941944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 85/166] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:34.942762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.071081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dde3c-a0a9-4ce6-85e6-6c1232a4a7b7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:35.099130 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aa68f-4399-4cdb-83da-bab3ca81fbe2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:35.120000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:35.120622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.120902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.121178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.121857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:35.122156 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:35.123561 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:35.136456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 86/167] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:35.144306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.148720 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:35.153534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:35.154038 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.154315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.154588 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.204591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:35.260118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:35.261624 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:35.300951 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8e187-4246-4688-ba94-39ee76a5a6de None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:35.353622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:35.354176 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.354453 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.354724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.359499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:35.361204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:35.362795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 87/168] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:35 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:35.364628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.426423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 82/169] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:35.427453 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.445649 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:35.517869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea368634-eafd-4fdf-8183-da173034b10c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:35.570334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:35.572858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:35.574206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.575656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.576458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.612759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64642a7f-00b3-43da-b0ab-7ebd6c27094e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:35.614168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:35.619863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:35.620739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:35.621520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:35.631622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 88/170] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:35.631622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.658354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6a2ec-c927-40f0-9f1e-2d16edc5837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:35.659132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6a2ec-c927-40f0-9f1e-2d16edc5837e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.659451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6a2ec-c927-40f0-9f1e-2d16edc5837e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.659714 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6a2ec-c927-40f0-9f1e-2d16edc5837e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.739355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6a2ec-c927-40f0-9f1e-2d16edc5837e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:35.741349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6a2ec-c927-40f0-9f1e-2d16edc5837e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:35.801824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add7682-080c-4b45-b5d9-256bc41b5a1b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:35.827222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:35.835744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:35.842438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2728a56-b07d-41de-bf4a-b07abf492bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:35.843380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728a56-b07d-41de-bf4a-b07abf492bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.843881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728a56-b07d-41de-bf4a-b07abf492bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.844500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728a56-b07d-41de-bf4a-b07abf492bc0 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.845939 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2728a56-b07d-41de-bf4a-b07abf492bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:35.847627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2728a56-b07d-41de-bf4a-b07abf492bc0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:35.849540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 89/171] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:35 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:35.850948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf266-94d6-495a-9f48-914b8daadd57 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.851478 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf266-94d6-495a-9f48-914b8daadd57 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.851976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf266-94d6-495a-9f48-914b8daadd57 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.853410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 90/172] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:35.858528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cf266-94d6-495a-9f48-914b8daadd57 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.878751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 83/173] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:35.881007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57643a-915c-4510-9670-1aac97c2833f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:35.881299 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57643a-915c-4510-9670-1aac97c2833f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:35.881579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57643a-915c-4510-9670-1aac97c2833f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:35.887881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 84/174] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:35.887881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57643a-915c-4510-9670-1aac97c2833f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:35.980766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cf266-94d6-495a-9f48-914b8daadd57 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.000358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57643a-915c-4510-9670-1aac97c2833f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.018336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.019380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.020434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.020907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.021798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:36.022396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:36.023988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1f6a-e4d8-485f-a59c-d5d5eb426528 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:36.033638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 91/175] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:36.035089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c3e6e-86bf-4950-a6a2-dc2159379784 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.035685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c3e6e-86bf-4950-a6a2-dc2159379784 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.036160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c3e6e-86bf-4950-a6a2-dc2159379784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.041906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.042587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.042864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.043133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.044050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] RBAC: Adding query filter params (role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=ff702c07bb134279b453ea6ea09d38b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:36.044359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:36.046147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:36.053014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 85/176] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:35 2022] GET /identity/v3/role_assignments?user.id=ff702c07bb134279b453ea6ea09d38b2&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:36.053896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:36.057250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-460c3e6e-86bf-4950-a6a2-dc2159379784 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:36.058818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-460c3e6e-86bf-4950-a6a2-dc2159379784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:36.089660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:36.140120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:36.152771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 92/177] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:36.154169 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.159976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.160958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.173158 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e5960-348f-47a7-89f5-d9a2b99fea33 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.192199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:36.193852 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:36.216270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.217247 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.217673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.218106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.224505 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:36.226290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:36.228321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 86/178] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:36 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:36.229439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:36.313414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 93/179] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:36.314479 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:36.365641 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a741b-d24e-4ea4-928c-4da168678a76 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.367746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:36.413771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.414348 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.414631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.414903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] PATH_INFO: `/v3/users/ff702c07bb134279b453ea6ea09d38b2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.416196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] RBAC: Authorizing `identity:get_user(user_id=ff702c07bb134279b453ea6ea09d38b2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:36.417726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:36.423536 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 87/180] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:36 2022] GET /identity/v3/users/ff702c07bb134279b453ea6ea09d38b2 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:36.425697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:36.540769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc86c86-97e7-44f9-9ad5-b07db88550e3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.547452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3447939-c09f-4e7a-af23-2bb8434ab88a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.582758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.583252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.583584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.583866 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.586829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:36.604225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.605283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.605764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.606253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.607371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=ff702c07bb134279b453ea6ea09d38b2) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:36.608080 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:36.609786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec2de6-f6b1-43ba-917a-949f9a1ea003 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:36.617533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 94/181] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:36 2022] GET /identity/v3/role_assignments?user.id=ff702c07bb134279b453ea6ea09d38b2&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=f2e63848ed864ae5901749a96467215f => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:36.619438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10134040-3fec-4638-9b8e-72ab87ddaf13 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.619971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10134040-3fec-4638-9b8e-72ab87ddaf13 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.620443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10134040-3fec-4638-9b8e-72ab87ddaf13 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.622533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 95/182] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:36.624275 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aa348-56d2-48c4-af98-69908e493f68 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.624989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:36.625424 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aa348-56d2-48c4-af98-69908e493f68 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.626755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aa348-56d2-48c4-af98-69908e493f68 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.628930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 96/183] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:36.640295 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975497-00f4-4674-8963-8fa8a8531ba9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.641016 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975497-00f4-4674-8963-8fa8a8531ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.641555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975497-00f4-4674-8963-8fa8a8531ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.669087 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:36.670876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:36.679836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8e975497-00f4-4674-8963-8fa8a8531ba9 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:36.681415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8e975497-00f4-4674-8963-8fa8a8531ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:36.780993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 97/184] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:36.782994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.783591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.784073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.798736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 88/185] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:36.799826 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:36.812313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:36.814675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:36.874729 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:36.928409 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7bd2c-057c-45d9-b24a-3cd6a5510d56 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:36.928944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 98/186] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:36.930097 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:36.982957 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:36.985623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:36.985623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:36.986652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:36.989788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.016608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:37.023774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:37.024659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:37.025438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 89/187] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:37.026087 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.073724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e3dda-9f25-4b63-a348-cf0f7757eb9f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.110999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.111654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.111924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.112190 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] PATH_INFO: `/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.113338 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] RBAC: Authorizing `identity:get_project(project_id=e216823ad4974e0eb3ad961d3bbc0b1c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.114848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.116584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 99/188] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:36 2022] GET /identity/v3/projects/e216823ad4974e0eb3ad961d3bbc0b1c => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.117354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.190129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9e253-55c6-4d21-82c5-a946136845e6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.224725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.234581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.235234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.235800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.236076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.242776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647c27b-0fdd-4d39-8efe-ca9160de7dce None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.246943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:37.251901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:37.252775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 90/189] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:37.259512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.265247 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.286681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.287312 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.287592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.287861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.288523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:37.288827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.290195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.298005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 100/190] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.300333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.352965 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03efad-a852-4462-9338-d19f65b40ed3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.386688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.387309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.387598 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.387874 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.388602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=e216823ad4974e0eb3ad961d3bbc0b1c, user.id=ff702c07bb134279b453ea6ea09d38b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:37.388915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.390347 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.396035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 91/191] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/role_assignments?user.id=ff702c07bb134279b453ea6ea09d38b2&scope.project.id=e216823ad4974e0eb3ad961d3bbc0b1c&role.id=f2e63848ed864ae5901749a96467215f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.396814 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.408822 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b8ba9-05fc-430b-a11d-fe1f08489f34 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.448389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.448952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.449232 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.449504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.454288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.462283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:37.467751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:37.468680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:37.469481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 101/192] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:37.474940 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.515063 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b09f5-1642-4506-ba40-cc5056bb2656 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.516352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.555409 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.555954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.556238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.557210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.557210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:37.557513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.558926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.568491 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 92/193] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.570240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.591727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f3e1-26b4-482e-8eaa-b49de2d87da2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.633339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.634269 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.634558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.635032 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.636502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.637936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.640152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 102/194] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.640751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.690941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb953c0c-6fc4-4288-a901-c1962142649e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.704571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.727884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.728523 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.728799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.729069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] PATH_INFO: `/v3/users/c03e9f74082d44c699e6257bbeb157ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.742858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b45509-328a-49a8-a141-96e8c6fd8c31 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.742993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] RBAC: Authorizing `identity:get_user(user_id=c03e9f74082d44c699e6257bbeb157ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.744457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.745604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.746055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 93/195] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/users/c03e9f74082d44c699e6257bbeb157ff => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.746942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.777823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.778428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.778706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.778981 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.779673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:37.779979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.781334 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.789658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 103/196] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.790429 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.849951 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8e0f8-b1a1-4a06-8c96-8b7b8a9a42c0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.882399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.883018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.883297 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.883589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] PATH_INFO: `/v3/users/9535d88e0c414427b19e4c50f73d786e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.884694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] RBAC: Authorizing `identity:get_user(user_id=9535d88e0c414427b19e4c50f73d786e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.886116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.886570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9186f5-5908-4ce7-974e-f64b80dc7987 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:37.887847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 94/197] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/users/9535d88e0c414427b19e4c50f73d786e => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:37.888444 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.924627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:37.925353 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:37.925634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:37.925909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:37.926994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.927113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:37.928572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:37.956617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 104/198] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:37 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:37.958869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:37.992132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:37.995978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2775569-fda4-4767-9408-22c6f8108d2b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.037070 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.037703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.038000 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.038255 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.086077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98350501-1c4e-4284-9361-4f3541d67899 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.122675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.123357 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.123645 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.124122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] PATH_INFO: `/v3/users/c03e9f74082d44c699e6257bbeb157ff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.125368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] RBAC: Authorizing `identity:get_user(user_id=c03e9f74082d44c699e6257bbeb157ff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.127329 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.127633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.129630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 105/199] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/users/c03e9f74082d44c699e6257bbeb157ff => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:38.129766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.129843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.176894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.241074 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.253364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74481f-7d83-478d-930e-edbcce4753d3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.256108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 95/200] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:38.257563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.306394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.307043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.307430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.307704 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.377578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571f421-94fe-4bcb-aeea-a59e21f3beda None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.386449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.388354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.426224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.426896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.427182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.428789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.438363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.492589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.505512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 106/201] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:38.511145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.524574 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.526808 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.628791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d8dc-031c-4b5a-abe6-b4c7fee0d4e7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.653441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 96/202] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:38.654618 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.678083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.678830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.679183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.679567 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.681469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.683304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.685146 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 107/203] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:38.686459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.708333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.754790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.786168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b5cda-fe64-439d-86e7-1c5696996326 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.809729 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68f3da-5f41-4aa9-aea1-d7854d5d3a5c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.825577 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.826254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.826537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.826974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.828379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.830520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.832911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 97/204] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:38.836047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.856283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:38.857210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:38.857498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:38.857774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:38.859136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:38.861003 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:38.863006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 108/205] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:38.864497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:38.948745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b853-d633-4948-af87-2d05e0c138c1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:38.959633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:38.993823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbcbfb-7b27-4b43-8a2f-7231f2ce1a06 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:39.000707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:39.001663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:39.002131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:39.002589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:39.003581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=c03e9f74082d44c699e6257bbeb157ff) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:39.004922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:39.007033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:39.015125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 98/206] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/role_assignments?user.id=c03e9f74082d44c699e6257bbeb157ff&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:39.016371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:39.027035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:39.039199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:39.040030 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:39.040445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:39.040843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:39.041760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] RBAC: Adding query filter params (role.id=7561f8727a4348488f94576d15c61021, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b, user.id=9535d88e0c414427b19e4c50f73d786e) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:39.042090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:39.043676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5df95-4418-4cba-a32f-0e0758d3ca8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:39.055502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 109/207] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:38 2022] GET /identity/v3/role_assignments?user.id=9535d88e0c414427b19e4c50f73d786e&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=7561f8727a4348488f94576d15c61021 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:39.163900 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9073ea-0535-44a2-97d7-cc7f2590de34 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:39.205062 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:39.205701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:39.205979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:39.206252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:39.207072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] RBAC: Adding query filter params (role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=c03e9f74082d44c699e6257bbeb157ff) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:39.207448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:39.208998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebdd02-7e9a-48bc-8812-3164b975fb9f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:39.215606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 99/208] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:39 2022] GET /identity/v3/role_assignments?user.id=c03e9f74082d44c699e6257bbeb157ff&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:40.342078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2db95-5549-43fb-9d69-fc21f81aefb5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:40.342585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2db95-5549-43fb-9d69-fc21f81aefb5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:40.343140 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2db95-5549-43fb-9d69-fc21f81aefb5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:40.345318 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 110/209] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:40.352233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6022f-b9e6-4132-a2ca-86edb9f821b8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:40.352785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6022f-b9e6-4132-a2ca-86edb9f821b8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:40.353307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6022f-b9e6-4132-a2ca-86edb9f821b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:40.377862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-57e6022f-b9e6-4132-a2ca-86edb9f821b8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:40.379686 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-57e6022f-b9e6-4132-a2ca-86edb9f821b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:40.409754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:40.457770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 100/210] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:40.538666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1200-a356-4372-8e73-d332122f5ae6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:40.538995 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1200-a356-4372-8e73-d332122f5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:40.539204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1200-a356-4372-8e73-d332122f5ae6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:40.540584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 111/211] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:40.545722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:40.546013 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:40.546290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:40.565039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:40.566203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:40.633865 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 101/212] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:40.763704 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b1200-a356-4372-8e73-d332122f5ae6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:40.874923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b1200-a356-4372-8e73-d332122f5ae6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:40.908223 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:40.913960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:40.914659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:40.914944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:40.915220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:40.930004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:40.937178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:40.938076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:40.938839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 112/213] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:40.943951 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.004248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:41.015723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:41.024930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d2b22-e9e9-4d8e-85cd-c633ff0f11ff None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:41.072492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:41.074664 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9c8e9-f8a4-4d2c-91de-dc0aee0d1f45 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:41.076112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.076818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.078217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.079678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:41.081363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:41.084556 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 102/214] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:41.090072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.116426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:41.117324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.117796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.118243 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.119070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:41.119580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:41.121233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:41.132041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 113/215] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:41.137220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.206717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bf2cc-255a-4a10-a686-71ef4ae3b372 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:41.243541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:41.244541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.245034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.245497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.246404 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:41.248050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:41.254830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef355b48-a24c-4260-9d01-866c1bd24b8f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:41.269281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:41.283909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 103/216] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:41 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:41.297644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4779f-a86b-4b9c-8a65-5d63d1827852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:41.298313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4779f-a86b-4b9c-8a65-5d63d1827852 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.298594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4779f-a86b-4b9c-8a65-5d63d1827852 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.298867 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4779f-a86b-4b9c-8a65-5d63d1827852 None admin] PATH_INFO: `/v3/users/37bf3163609749308d2bb50ee3c4f17d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.315733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4779f-a86b-4b9c-8a65-5d63d1827852 None admin] RBAC: Authorizing `identity:get_user(user_id=37bf3163609749308d2bb50ee3c4f17d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:41.317751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4779f-a86b-4b9c-8a65-5d63d1827852 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:41.319738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 114/217] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:41 2022] GET /identity/v3/users/37bf3163609749308d2bb50ee3c4f17d => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:41.325325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.350200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:41.413262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c531-3c0f-4f0b-a6c8-e16ab8009b72 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.413771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c531-3c0f-4f0b-a6c8-e16ab8009b72 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.413771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c531-3c0f-4f0b-a6c8-e16ab8009b72 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.415270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 115/218] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:41.422896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b9b17-151c-4673-b8a0-f45a29f67305 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.423560 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b9b17-151c-4673-b8a0-f45a29f67305 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.424054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b9b17-151c-4673-b8a0-f45a29f67305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.448829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8d3b9b17-151c-4673-b8a0-f45a29f67305 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:41.451259 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8d3b9b17-151c-4673-b8a0-f45a29f67305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:41.452746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f183fc-2447-4174-ac6d-ef7666cbaf94 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:41.492122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:41.492658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.493128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.493247 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.532420 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 116/219] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:41.533609 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b0ec6-c3ce-4b99-a842-4f1331f39a78 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.533892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b0ec6-c3ce-4b99-a842-4f1331f39a78 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.534165 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b0ec6-c3ce-4b99-a842-4f1331f39a78 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.536248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 117/220] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:41.540752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.541031 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.541305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.559982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:41.561149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:41.585804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:41.587905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:41.609379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:41.644867 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 118/221] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:41.704324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 104/222] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:41.710090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.744787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:41.822142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959cd02-e0c0-4648-b9b9-b0677451787d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:41.841605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.874716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:41.875749 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:41.876545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:41.877501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:41.879746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:41.881202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:41.882979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 119/223] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:41 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:41.889579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:41.991179 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177acbc-198d-454d-b57f-52a074c92320 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.010649 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:42.030033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.031008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.031520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.031990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.032732 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:42.037592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71962d3d-c5b8-4f72-9d60-49d27664da37 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.046997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:42.054011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:42.054894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:42.055741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 105/224] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:42.056854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.083394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.084203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.084657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.085121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.086119 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=37bf3163609749308d2bb50ee3c4f17d) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:42.086580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.088312 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.094866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 120/225] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:41 2022] GET /identity/v3/role_assignments?user.id=37bf3163609749308d2bb50ee3c4f17d&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=f2e63848ed864ae5901749a96467215f => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:42.096145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.216024 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3bb82-c34e-4bf0-9d5d-71c598b951fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.218987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489ad6-6162-474b-8b4f-8fdac3777b0b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.252712 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.253797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.254810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.255276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.268118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:42.274473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:42.275366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:42.276168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:42.278956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 106/226] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:42.287343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.287343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.287343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.287343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.287743 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:42.287743 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.288356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.289322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.298455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 121/227] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:42.308610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.339589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:42.374443 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:42.444903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4f76-3718-4395-b65a-8dc0ecc2e468 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.491597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.492240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.492523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.492797 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] PATH_INFO: `/v3/users/c03e9f74082d44c699e6257bbeb157ff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.494536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] RBAC: Authorizing `identity:get_user(user_id=c03e9f74082d44c699e6257bbeb157ff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.496035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.497641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 122/228] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/users/c03e9f74082d44c699e6257bbeb157ff => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:42.505186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.510370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b904b97-3b37-4dec-8837-16014e851c52 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.581915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.582983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.583272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.583570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.584250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:42.584559 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.586322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.596784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 107/229] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:42.604292 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.643663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:42.652530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23362369-6a59-46bc-a11c-eec1b879e7cd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.689955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.690600 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.690881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.691159 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.693137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.694731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.701477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 123/230] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:42.702800 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.808990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273cd3-b8a4-452b-b626-581fdf2e20ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.844263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.844973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.845386 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:42.845890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.846170 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/c03e9f74082d44c699e6257bbeb157ff/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.849443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] RBAC: Authorizing `identity:create_grant(project_id=015ebad4cd344e7f8b6506b57b7a7b90, user_id=c03e9f74082d44c699e6257bbeb157ff, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.850902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.857721 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabc015-b267-43df-b1e0-1ffd63019ef0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:42.873665 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 124/231] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:42 2022] PUT /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/c03e9f74082d44c699e6257bbeb157ff/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:04:42.918157 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:42.918938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:42.919237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:42.919526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] PATH_INFO: `/v3/users/c03e9f74082d44c699e6257bbeb157ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:42.921827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] RBAC: Authorizing `identity:get_user(user_id=c03e9f74082d44c699e6257bbeb157ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:42.923328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:42.925107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 108/232] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/users/c03e9f74082d44c699e6257bbeb157ff => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:42.930679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:42.982790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:43.112526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1342e-fc7f-4e5e-9af8-b8c8afdff111 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:43.151755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fcc75-1d66-4c6d-8f2f-5b78625ed368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:43.152397 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fcc75-1d66-4c6d-8f2f-5b78625ed368 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:43.152680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fcc75-1d66-4c6d-8f2f-5b78625ed368 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:43.152955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fcc75-1d66-4c6d-8f2f-5b78625ed368 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:43.154093 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fcc75-1d66-4c6d-8f2f-5b78625ed368 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:43.155585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fcc75-1d66-4c6d-8f2f-5b78625ed368 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:43.157659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 125/233] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:42 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:43.163705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:43.280894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b93a-693d-45d2-84b6-3591de3100a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:43.325183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0e95f-fdc2-4e0c-ab25-eb0f2044ee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:43.326112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e95f-fdc2-4e0c-ab25-eb0f2044ee47 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:43.326599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e95f-fdc2-4e0c-ab25-eb0f2044ee47 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:43.327064 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e95f-fdc2-4e0c-ab25-eb0f2044ee47 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/c03e9f74082d44c699e6257bbeb157ff/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:43.331009 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0e95f-fdc2-4e0c-ab25-eb0f2044ee47 None admin] RBAC: Authorizing `identity:create_grant(project_id=015ebad4cd344e7f8b6506b57b7a7b90, user_id=c03e9f74082d44c699e6257bbeb157ff, role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:43.333254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0e95f-fdc2-4e0c-ab25-eb0f2044ee47 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:43.354531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 109/234] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:43 2022] PUT /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/c03e9f74082d44c699e6257bbeb157ff/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:04:44.070337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c79e12-3df8-4aff-872e-11e7d4643750 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.071335 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c79e12-3df8-4aff-872e-11e7d4643750 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.071835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c79e12-3df8-4aff-872e-11e7d4643750 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.074491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 126/235] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:44.079674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a49e6-301b-4d0b-b54d-1b67a38ea200 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.080211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a49e6-301b-4d0b-b54d-1b67a38ea200 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.080211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a49e6-301b-4d0b-b54d-1b67a38ea200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.100397 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dc1a49e6-301b-4d0b-b54d-1b67a38ea200 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:44.101603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dc1a49e6-301b-4d0b-b54d-1b67a38ea200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:44.126261 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:44.172462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 110/236] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:44.219306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206920-a599-4f2b-a2e9-d36bf2a00fab None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.219306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206920-a599-4f2b-a2e9-d36bf2a00fab None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.219306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206920-a599-4f2b-a2e9-d36bf2a00fab None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.219904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 127/237] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:44.225445 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.225925 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.226341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.246879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:44.248444 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:44.321405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 111/238] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:44.446405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206920-a599-4f2b-a2e9-d36bf2a00fab None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:44.542958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:44.559175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206920-a599-4f2b-a2e9-d36bf2a00fab None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:44.593481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:44.600869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:44.601504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.601843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.602117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.603217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:44.604694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:44.606602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 128/239] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:44.615310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:44.695805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:44.712219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1672c-08d9-400c-bbcc-e348b1f7dff2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:44.714967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81818b9-361f-43a3-b36d-502d78715814 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:44.749698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271ba21-919d-4d39-9521-ebabea71df5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:44.749926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef028e93-2c15-456b-a931-b1434a007e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:44.750304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271ba21-919d-4d39-9521-ebabea71df5e None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.750406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028e93-2c15-456b-a931-b1434a007e87 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.750581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271ba21-919d-4d39-9521-ebabea71df5e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.750676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028e93-2c15-456b-a931-b1434a007e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.750856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028e93-2c15-456b-a931-b1434a007e87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.750949 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271ba21-919d-4d39-9521-ebabea71df5e None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.751725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271ba21-919d-4d39-9521-ebabea71df5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:44.753218 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271ba21-919d-4d39-9521-ebabea71df5e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:44.761739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef028e93-2c15-456b-a931-b1434a007e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ef028e93-2c15-456b-a931-b1434a007e87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:44.767456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:44.768322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:44.769534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 112/240] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:44.775026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117586c-9fce-4456-9085-a9217af5f435 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.775496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117586c-9fce-4456-9085-a9217af5f435 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.775942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117586c-9fce-4456-9085-a9217af5f435 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.778203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 113/241] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:44.778843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117586c-9fce-4456-9085-a9217af5f435 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:44.780613 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 129/242] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:44 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:44.789685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.789994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.790301 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.824655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:44.825870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:44.840951 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:44.904253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117586c-9fce-4456-9085-a9217af5f435 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:44.910641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 130/243] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:44.943976 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:44.944843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:44.945266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:44.945679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:44.946529 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:44.946979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:44.948572 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:44.958926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 114/244] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:44.964941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.087078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091505-30f4-4ca4-8dcf-76f0081243e0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.126145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.127151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.127472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.127754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] PATH_INFO: `/v3/users/619eca644e3c46419963c4c33ae00ab8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.140055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:45.144738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] RBAC: Authorizing `identity:get_user(user_id=619eca644e3c46419963c4c33ae00ab8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:45.146522 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:45.148271 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 131/245] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:44 2022] GET /identity/v3/users/619eca644e3c46419963c4c33ae00ab8 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:45.153870 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.167592 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:45.271366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fd16c-6980-4c14-aa4f-934a8fc5a28a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.280871 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.312504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.313125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.313405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.313674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.391197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:45.392698 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:45.399063 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a6490-4fa1-4c0a-aedd-f64233892d2b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.405163 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:45.436192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.436771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.437053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.437329 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.448687 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:45.453636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:45.455890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:45.457938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 132/246] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:45.468534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.502202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 115/247] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:45.507932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.541049 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:45.592835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1631c-5845-4291-ae55-6954192c24e3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.628864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764008c-4310-42c4-ac56-84b369d8507d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.640522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:45.649289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.649289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.649520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.649784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.650467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:45.650774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:45.652722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:45.667490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 133/248] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:45.672272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.686945 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.687731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.688017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.688348 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.690439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:45.692872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:45.695566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 116/249] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:45.701725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.822585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ff7f8-7b1a-46eb-b364-e45c5fa6120c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.828115 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772951b-8fb8-42b0-b33e-d3111f523f58 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:45.851585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:45.864737 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.865405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.865648 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.865922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] PATH_INFO: `/v3/users/37bf3163609749308d2bb50ee3c4f17d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.867042 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] RBAC: Authorizing `identity:get_user(user_id=37bf3163609749308d2bb50ee3c4f17d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:45.868511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:45.870376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 134/250] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/users/37bf3163609749308d2bb50ee3c4f17d => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:45.880623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:45.886625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:45.887448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:45.887891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:45.889205 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:45.890110 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] RBAC: Adding query filter params (role.id=7561f8727a4348488f94576d15c61021, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=619eca644e3c46419963c4c33ae00ab8) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:45.893748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:45.895535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:45.907990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 117/251] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/role_assignments?user.id=619eca644e3c46419963c4c33ae00ab8&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=7561f8727a4348488f94576d15c61021 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:46.013720 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a10b4-664c-40c6-8f6d-767c86324807 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:46.059028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38153f-40f9-47d3-8165-b2f2f8ccaeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:46.060474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38153f-40f9-47d3-8165-b2f2f8ccaeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:46.060960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38153f-40f9-47d3-8165-b2f2f8ccaeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:46.061467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38153f-40f9-47d3-8165-b2f2f8ccaeb8 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:46.068278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38153f-40f9-47d3-8165-b2f2f8ccaeb8 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:46.070139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38153f-40f9-47d3-8165-b2f2f8ccaeb8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:46.072326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 135/252] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:45 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:46.079353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:46.150195 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:46.201251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfaceb-ea49-4ca6-b2cc-05be49c6bb17 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:46.201755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfaceb-ea49-4ca6-b2cc-05be49c6bb17 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:46.201828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfaceb-ea49-4ca6-b2cc-05be49c6bb17 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:46.203123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 136/253] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:46.209253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d379-5980-4fef-9a71-cb23b7e8e143 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:46.209536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d379-5980-4fef-9a71-cb23b7e8e143 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:46.209812 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208d379-5980-4fef-9a71-cb23b7e8e143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:46.233728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a208d379-5980-4fef-9a71-cb23b7e8e143 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:46.234954 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a208d379-5980-4fef-9a71-cb23b7e8e143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:46.249857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef3a62-5aa2-4248-8863-061b0647aec6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:46.323554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73d798-4425-4b9c-b0f7-9423283cf913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:46.324799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73d798-4425-4b9c-b0f7-9423283cf913 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:46.325091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73d798-4425-4b9c-b0f7-9423283cf913 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:46.325370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73d798-4425-4b9c-b0f7-9423283cf913 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/37bf3163609749308d2bb50ee3c4f17d/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:46.327723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73d798-4425-4b9c-b0f7-9423283cf913 None admin] RBAC: Authorizing `identity:create_grant(project_id=015ebad4cd344e7f8b6506b57b7a7b90, user_id=37bf3163609749308d2bb50ee3c4f17d, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:46.329174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73d798-4425-4b9c-b0f7-9423283cf913 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:46.349387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:46.359666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 118/254] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:46 2022] PUT /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/37bf3163609749308d2bb50ee3c4f17d/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:46.382809 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 137/255] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:46.637948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ed19-7e36-46da-bf03-025a57b7b0f5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:46.638292 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ed19-7e36-46da-bf03-025a57b7b0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:46.638479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ed19-7e36-46da-bf03-025a57b7b0f5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:46.639832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 119/256] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:46.645718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:46.645981 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:46.646257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:46.675547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:46.676847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:46.750195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 138/257] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:46.837010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ed19-7e36-46da-bf03-025a57b7b0f5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:46.978832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ed19-7e36-46da-bf03-025a57b7b0f5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:46.980264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:47.027615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:47.028255 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:47.028534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:47.028812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:47.042920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:47.047923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:47.048783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:47.049565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 120/258] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:47.055318 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:47.216672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc2ef0-4c20-43cb-981e-c150bef07f78 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:47.244618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:47.264740 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:47.265354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:47.265622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:47.265980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:47.266644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:47.266937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:47.267130 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:47.268337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:47.283142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 139/259] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:47.294535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:47.449677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf2f2-c9b9-446e-95e9-5916b23f8466 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:47.503179 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb544819-dec6-4144-b894-b7bbb9d66b78 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:47.512877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:47.513633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:47.513921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:47.514197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:47.539823 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:47.546697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:47.547611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:04:47.548461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:47.555920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 121/260] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:04:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:47.562419 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:47.592000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:47.592853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:47.593127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:47.593393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] PATH_INFO: `/v3/users/c03e9f74082d44c699e6257bbeb157ff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:47.595897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] RBAC: Authorizing `identity:get_user(user_id=c03e9f74082d44c699e6257bbeb157ff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:47.597807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:47.600582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 140/261] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:47 2022] GET /identity/v3/users/c03e9f74082d44c699e6257bbeb157ff => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:47.606691 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:47.621252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:47.729133 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:47.757978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cef29-4375-4ce0-a350-8fd6fbb53d5a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:47.818745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57db1cf-37a5-42d3-ad22-e0ba34d4640e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:47.823176 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:47.823974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:47.824555 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:47.825028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:47.825919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:47.826421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:47.828043 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:47.849055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 122/262] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:04:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:47.855765 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:47.916395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:47.917294 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:47.917708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:47.918108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.017924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:48.053481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137a0ab-c94e-4bc8-a750-547553344b94 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:48.097361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:48.099380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:48.142283 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:48.142942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.143229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.143538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] PATH_INFO: `/v3/users/c03e9f74082d44c699e6257bbeb157ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.144720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] RBAC: Authorizing `identity:get_user(user_id=c03e9f74082d44c699e6257bbeb157ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:48.146747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:48.151395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 123/263] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:47 2022] GET /identity/v3/users/c03e9f74082d44c699e6257bbeb157ff => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:48.162448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:48.195728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:48.251781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 141/264] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 19:04:48.261830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:48.375304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa832200-1f5d-4894-bb45-7132b1f8d78f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:48.440535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:48.441502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.442015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.442485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.461484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0784291-5231-4c9a-9a41-fe4b18f5f985 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:48.483673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:48.507175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:48.507843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.508120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.508388 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.509556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:48.510994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:48.512972 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 142/265] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:48 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:48.518511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:48.556555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:48.569533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:48.572040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:48.671803 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402daa7-8d36-41d1-a60c-8a879bdf3400 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:48.756066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:48.756694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.756975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.757250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.757997 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=c03e9f74082d44c699e6257bbeb157ff) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:48.758302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:48.759782 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3dc92-d9c7-4eb9-8696-8fe0073b8bf2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:48.766629 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 143/266] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:48 2022] GET /identity/v3/role_assignments?user.id=c03e9f74082d44c699e6257bbeb157ff&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:48.778277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b66bc5-6c46-498d-805b-a0db9ee05c5f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.778578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b66bc5-6c46-498d-805b-a0db9ee05c5f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.778818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b66bc5-6c46-498d-805b-a0db9ee05c5f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.780570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 144/267] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:48.788590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9fbd7-10eb-498d-8055-6446e0ce2677 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.788905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9fbd7-10eb-498d-8055-6446e0ce2677 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.789129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9fbd7-10eb-498d-8055-6446e0ce2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.793287 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 124/268] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:48.798704 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:48.826124 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-35c9fbd7-10eb-498d-8055-6446e0ce2677 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:48.827356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-35c9fbd7-10eb-498d-8055-6446e0ce2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:48.923772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:48.981245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 145/269] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:48.982573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8248b59-bf8f-41f1-a4e1-68367a1456d0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.982908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8248b59-bf8f-41f1-a4e1-68367a1456d0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.983184 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8248b59-bf8f-41f1-a4e1-68367a1456d0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:48.984630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 146/270] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:48.991500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e78701-48c4-477d-90da-35f6232dfbc4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:48.992098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e78701-48c4-477d-90da-35f6232dfbc4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:48.992578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e78701-48c4-477d-90da-35f6232dfbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.024147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-11e78701-48c4-477d-90da-35f6232dfbc4 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:49.025347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11e78701-48c4-477d-90da-35f6232dfbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:49.031614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:49.047671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1de0-cc38-4049-bc71-fc97e557541f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.106428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 147/271] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:49.107968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c98ab-e28f-4e4d-9b31-c0a90a87547d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.108501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c98ab-e28f-4e4d-9b31-c0a90a87547d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.108782 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c98ab-e28f-4e4d-9b31-c0a90a87547d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.111177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 148/272] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:49.114267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.115089 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.115266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.115560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.116294 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a82027-acd4-4715-82c0-71f572330caa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.116575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a82027-acd4-4715-82c0-71f572330caa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.116686 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:49.116838 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a82027-acd4-4715-82c0-71f572330caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.118101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:49.120533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 125/273] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:48 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:49.137221 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.140648 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-77a82027-acd4-4715-82c0-71f572330caa None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:49.142861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-77a82027-acd4-4715-82c0-71f572330caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:49.234090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 149/274] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:49.256720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba34d32-672d-4719-9d91-c907fd21e8be None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.279025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:49.293619 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.294280 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.294562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.294834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.295588 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] RBAC: Adding query filter params (role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=c03e9f74082d44c699e6257bbeb157ff) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:49.295899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:49.297410 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:49.304099 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 126/275] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/role_assignments?user.id=c03e9f74082d44c699e6257bbeb157ff&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:49.466594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a82027-acd4-4715-82c0-71f572330caa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.516672 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.574155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:49.610757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a82027-acd4-4715-82c0-71f572330caa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.636688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdc941-d71f-4b15-ad4a-096ffa6f9ccf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.651102 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.651772 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.652047 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.652317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.664892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:49.669742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:49.670615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:49.671416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 150/276] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:49.672141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.681818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a8b4b-f318-4f20-9ef7-65b136653898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.682710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8b4b-f318-4f20-9ef7-65b136653898 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.683225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8b4b-f318-4f20-9ef7-65b136653898 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.683800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a8b4b-f318-4f20-9ef7-65b136653898 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.696407 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a8b4b-f318-4f20-9ef7-65b136653898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-914a8b4b-f318-4f20-9ef7-65b136653898 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:49.701553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:49.702424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:49.703232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 127/277] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:49.703952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29aca9-bafb-43af-87b3-53e6a4a2e49e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.704183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29aca9-bafb-43af-87b3-53e6a4a2e49e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.704399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29aca9-bafb-43af-87b3-53e6a4a2e49e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.705634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 128/278] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:49.706326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29aca9-bafb-43af-87b3-53e6a4a2e49e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.753039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:49.781405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc43e7-1e4a-4eb3-a71b-6a765d4ebe15 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.807795 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29aca9-bafb-43af-87b3-53e6a4a2e49e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.808598 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:49.814693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.815340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.815615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.815883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.817518 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:49.818948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:49.820515 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 151/279] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:49.821271 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.842318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.842925 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.843204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.843797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.844168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:49.844474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:49.845838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48228ae-ce3c-4238-b690-cbac5183854f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:49.854799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 129/280] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:49.855996 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.856272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.856540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.873536 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:49.874720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:49.924578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa95f54-f09b-446d-9f36-ecf3c32505d2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:49.939754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 130/281] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:49.940561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.961058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:49.961652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:49.962342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:49.962618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:49.963573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:49.963877 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:49.965273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:49.974058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 152/282] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:49.976229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:49.980160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.039086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.056326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8aae22-bc71-4b93-b30f-4daa20a4b1c6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.087374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d242fc2-87f5-49d5-a4af-6caf83a09c66 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.093518 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.094129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.094407 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.094922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.095364 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.096812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.121399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 131/283] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:04:49 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:50.122316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.125644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.126292 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.126569 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.126837 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] PATH_INFO: `/v3/users/2c1ef9d5602f428484b1b89503f10c3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.151674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1ef9d5602f428484b1b89503f10c3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.153327 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.155667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 153/284] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:49 2022] GET /identity/v3/users/2c1ef9d5602f428484b1b89503f10c3c => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:50.162395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.249103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.262862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b345d-976b-4f3b-bd4d-c956d64db4f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.291392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f3b62-e7f0-42fb-8edc-4eb021ac8940 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.303751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.304598 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.304879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.305149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] PATH_INFO: `/v3/users/619eca644e3c46419963c4c33ae00ab8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.306277 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] RBAC: Authorizing `identity:get_user(user_id=619eca644e3c46419963c4c33ae00ab8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.307746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.309318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 132/285] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/users/619eca644e3c46419963c4c33ae00ab8 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:50.310074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.310880 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.335924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.336827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.337273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.337735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.338689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, user.id=2c1ef9d5602f428484b1b89503f10c3c, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:50.339178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.340630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.346231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 154/286] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/role_assignments?user.id=2c1ef9d5602f428484b1b89503f10c3c&scope.system=all&role.id=f2e63848ed864ae5901749a96467215f => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:50.347109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.465514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4a961-6710-4f55-8e3e-c1e8e3c712e8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.479347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe8940-25d1-4bef-89bb-eff4810c9bd8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.516341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.517061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.517597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.517597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.527132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.528025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.528684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.528986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.530264 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.531711 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:50.532084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.533947 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 155/287] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:50.538801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:50.539694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:50.540487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 133/288] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:50.541126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.547903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.570539 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.605589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.665158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44259a50-9112-4559-bb0b-79f7b145a77b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.666717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295edb33-2b5d-4e28-8713-50e4c2c9d876 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.700407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.701182 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.701464 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.701740 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/619eca644e3c46419963c4c33ae00ab8/roles/7561f8727a4348488f94576d15c61021` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.704107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=015ebad4cd344e7f8b6506b57b7a7b90, user_id=619eca644e3c46419963c4c33ae00ab8, role_id=7561f8727a4348488f94576d15c61021)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.705278 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.705629 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.706322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.706804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.707266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.708782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:50.709392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.711082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.720447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 134/289] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:50.726289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:50.737723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 156/290] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:04:50 2022] PUT /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/users/619eca644e3c46419963c4c33ae00ab8/roles/7561f8727a4348488f94576d15c61021 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:50.819639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:50.840362 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb14fd-e06f-4486-a856-ae236dd56034 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:50.886292 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:50.886956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:50.887237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:50.887544 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] PATH_INFO: `/v3/users/37bf3163609749308d2bb50ee3c4f17d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:50.888853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] RBAC: Authorizing `identity:get_user(user_id=37bf3163609749308d2bb50ee3c4f17d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:50.890396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:50.892131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 135/291] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/users/37bf3163609749308d2bb50ee3c4f17d => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:50.898416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:51.009781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db80a4-de5f-4e66-bd03-fae41b7c00d1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:51.021828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:51.047107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:51.047752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:51.048031 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:51.048302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:51.119869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:51.121402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:51.223823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 157/292] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:51.229518 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:51.355267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e8451-1beb-4bde-85eb-db43ac15de23 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:51.392496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c7679-e5f7-407a-affa-2e114d85a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:51.393193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7679-e5f7-407a-affa-2e114d85a182 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:51.393476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7679-e5f7-407a-affa-2e114d85a182 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:51.393745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c7679-e5f7-407a-affa-2e114d85a182 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:51.394909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c7679-e5f7-407a-affa-2e114d85a182 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:51.396408 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c7679-e5f7-407a-affa-2e114d85a182 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:51.398342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 136/293] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:51 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:51.404077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:51.416392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:51.524059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddcfb3-fcb0-4ce7-bbba-c748b4d80f60 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:51.558563 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:51.559305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:51.559603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:51.559876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:51.560601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=37bf3163609749308d2bb50ee3c4f17d) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:51.560906 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:51.562337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc75df-00f1-478c-abe6-b6dc1a4b7520 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:51.568626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 158/294] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:51 2022] GET /identity/v3/role_assignments?user.id=37bf3163609749308d2bb50ee3c4f17d&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=f2e63848ed864ae5901749a96467215f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:52.758275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331ead-578b-444b-bfa6-cba8fc145ee5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:52.758822 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331ead-578b-444b-bfa6-cba8fc145ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:52.758822 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331ead-578b-444b-bfa6-cba8fc145ee5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:52.760238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 137/295] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:52.765542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680b4f7-3d1c-40a5-97f1-6f68007a9d0d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:52.765927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680b4f7-3d1c-40a5-97f1-6f68007a9d0d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:52.766083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680b4f7-3d1c-40a5-97f1-6f68007a9d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:52.785110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3680b4f7-3d1c-40a5-97f1-6f68007a9d0d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:52.786389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3680b4f7-3d1c-40a5-97f1-6f68007a9d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:52.839424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252dd42-025f-4b3c-9696-1f758c0f482d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:52.839810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252dd42-025f-4b3c-9696-1f758c0f482d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:52.840159 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252dd42-025f-4b3c-9696-1f758c0f482d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:52.841508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 138/296] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:52.847156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d20c8-b955-487c-973f-ccd3807e4173 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:52.847475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d20c8-b955-487c-973f-ccd3807e4173 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:52.847736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d20c8-b955-487c-973f-ccd3807e4173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:52.871495 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 159/297] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:52.872948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-999d20c8-b955-487c-973f-ccd3807e4173 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:52.874336 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-999d20c8-b955-487c-973f-ccd3807e4173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:52.890161 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:52.945358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 139/298] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:04:52.980048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae25a48-732f-4426-82a4-82ee90e99faf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:52.980330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae25a48-732f-4426-82a4-82ee90e99faf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:52.980600 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae25a48-732f-4426-82a4-82ee90e99faf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:52.982191 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 160/299] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:52.987596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:52.988055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:52.988526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.008441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:53.009901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:53.086165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 140/300] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:53.158076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae25a48-732f-4426-82a4-82ee90e99faf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.162393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.269446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.276696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae25a48-732f-4426-82a4-82ee90e99faf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.323066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.323963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.324453 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.324881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.337014 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:53.343415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:53.344428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:53.345222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 161/301] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:53.351714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.370787 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.401165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4e39e-fa95-414e-9e49-e0004448d9ea None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.421701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.437607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.438248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.438520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.438791 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.448751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:53.453658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:53.454535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:53.455378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 141/302] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:04:53.456194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.469424 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821b7c-a692-421d-b9f5-ec76a312261e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.503309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.503946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.504219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.504510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.505168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:53.505467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:53.506831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:53.514557 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 162/303] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:53.515351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.558011 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be130c6b-5c61-4fad-89c0-f71185cf2f02 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.590346 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a07b03-5586-49c0-9abb-af983a4ae181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.590963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07b03-5586-49c0-9abb-af983a4ae181 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.591238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07b03-5586-49c0-9abb-af983a4ae181 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.591530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07b03-5586-49c0-9abb-af983a4ae181 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.592626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a07b03-5586-49c0-9abb-af983a4ae181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:53.594045 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a07b03-5586-49c0-9abb-af983a4ae181 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:53.595605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 142/304] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:53.596690 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e23934-32c7-4425-8888-a0ac522a4496 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.596966 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e23934-32c7-4425-8888-a0ac522a4496 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.597253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e23934-32c7-4425-8888-a0ac522a4496 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.598955 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 143/305] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:53.599666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e23934-32c7-4425-8888-a0ac522a4496 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.607979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.620709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2645c8-6288-4829-91a3-4f36f136f0a7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.645589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.654940 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.655575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.655861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.656137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.656794 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:53.657106 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:53.658460 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:53.666717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 163/306] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:53.667509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.704701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e23934-32c7-4425-8888-a0ac522a4496 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.739179 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deeba8-f334-4ada-b735-977675aa709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.739924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deeba8-f334-4ada-b735-977675aa709f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.740300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deeba8-f334-4ada-b735-977675aa709f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.740580 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deeba8-f334-4ada-b735-977675aa709f None admin] PATH_INFO: `/v3/users/2c1ef9d5602f428484b1b89503f10c3c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.741703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8deeba8-f334-4ada-b735-977675aa709f None admin] RBAC: Authorizing `identity:get_user(user_id=2c1ef9d5602f428484b1b89503f10c3c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:53.743146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8deeba8-f334-4ada-b735-977675aa709f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:53.744780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 144/307] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/users/2c1ef9d5602f428484b1b89503f10c3c => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:53.745976 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.746259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.746532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.763794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:53.764984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:53.771938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691e72-db8b-4283-9f69-f6ab321f70a9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.805531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:53.806139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:53.806425 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:53.806699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] PATH_INFO: `/v3/groups` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:53.807346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:53.808823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:53.824207 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 164/308] 10.222.0.18 () {60 vars in 1134 bytes} [Fri Sep 16 19:04:53 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:53.825015 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.846931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 145/309] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:53.852308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:53.857343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.884593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:53.967593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ab319-dca0-4ab3-87d6-5367ccde21a8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:53.987385 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb5747-ccab-4430-88b0-b3bf9b2bbd8c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:54.017803 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.018439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.018718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.019257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] PATH_INFO: `/v3/users/6246d854c15a45f6bc7aa0f90873d2be` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.034055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.034600 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] RBAC: Authorizing `identity:get_user(user_id=6246d854c15a45f6bc7aa0f90873d2be)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:54.034703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.034991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.035268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] PATH_INFO: `/v3/system/users/2c1ef9d5602f428484b1b89503f10c3c/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.036226 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:54.036762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2c1ef9d5602f428484b1b89503f10c3c, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:54.038034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 165/310] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:53 2022] GET /identity/v3/users/6246d854c15a45f6bc7aa0f90873d2be => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:54.038210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:54.043712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:54.050226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 146/311] 10.222.0.18 () {58 vars in 1334 bytes} [Fri Sep 16 19:04:53 2022] PUT /identity/v3/system/users/2c1ef9d5602f428484b1b89503f10c3c/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:54.123631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:54.173659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d335b-1106-43d7-911d-0d5685ded506 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:54.185732 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:54.220371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:54.224850 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.226110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.227223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.227365 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.228196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] RBAC: Adding query filter params (role.id=7561f8727a4348488f94576d15c61021, user.id=6246d854c15a45f6bc7aa0f90873d2be, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:54.228544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:54.230369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:54.237673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 166/312] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Sep 16 19:04:54 2022] GET /identity/v3/role_assignments?user.id=6246d854c15a45f6bc7aa0f90873d2be&scope.system=all&role.id=7561f8727a4348488f94576d15c61021 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:54.263666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0557-4f91-47c7-9e3d-1da617b26fd4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:54.308178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.309178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.309674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.310133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.322896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:54.329193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:54.330091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:54.330871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:54.333800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 147/313] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:54.339908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:54.450143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44874924-801c-40f4-9ad5-9696e2196865 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:54.485354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.485968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.486507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.486780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.487474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:54.487780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:54.489675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:54.498541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 167/314] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:54.504638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:54.607743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f72283-418c-4b45-9346-76779ebb442c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:54.639441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:54.649299 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.650230 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.650720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.651261 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] PATH_INFO: `/v3/users/619eca644e3c46419963c4c33ae00ab8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.652633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] RBAC: Authorizing `identity:get_user(user_id=619eca644e3c46419963c4c33ae00ab8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:54.654261 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:54.656377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 148/315] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:54 2022] GET /identity/v3/users/619eca644e3c46419963c4c33ae00ab8 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:54.664370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:54.699276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:54.774089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03692804-85c6-48eb-aa28-33a326c049cc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:54.809281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:54.809896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:54.810173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:54.810446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:54.884552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:54.885942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:54.917136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:54.990280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 168/316] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:04:54.996263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:55.105268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cad330-dcb5-4fc4-9bc2-2fc59344ead7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:55.140265 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987ed77-1b8c-44a4-879b-a84fe0c08d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:55.140821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987ed77-1b8c-44a4-879b-a84fe0c08d5f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:55.141097 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987ed77-1b8c-44a4-879b-a84fe0c08d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:55.141366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987ed77-1b8c-44a4-879b-a84fe0c08d5f None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:55.142481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987ed77-1b8c-44a4-879b-a84fe0c08d5f None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:55.143934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987ed77-1b8c-44a4-879b-a84fe0c08d5f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:55.145648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 149/317] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:04:54 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:55.151719 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:55.259402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47aaa3-caa0-49b9-a33a-95a6a7e36184 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:55.291026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:55.295311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:55.296098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:55.296468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:55.296750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:55.297480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] RBAC: Adding query filter params (role.id=7561f8727a4348488f94576d15c61021, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90, user.id=619eca644e3c46419963c4c33ae00ab8) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:55.297791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:55.299208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785096fe-234d-464f-bd4c-c66a11210f03 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:55.305517 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 169/318] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:04:55 2022] GET /identity/v3/role_assignments?user.id=619eca644e3c46419963c4c33ae00ab8&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=7561f8727a4348488f94576d15c61021 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:56.123117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc8735-ddfe-46f8-a1a2-c809d474bc68 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.123691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc8735-ddfe-46f8-a1a2-c809d474bc68 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.123691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc8735-ddfe-46f8-a1a2-c809d474bc68 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.125061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 150/319] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:56.130255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52b045-05d7-4bc4-b2f9-4d8e1d481e6e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.130667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52b045-05d7-4bc4-b2f9-4d8e1d481e6e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.130838 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52b045-05d7-4bc4-b2f9-4d8e1d481e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.149148 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7b52b045-05d7-4bc4-b2f9-4d8e1d481e6e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:56.150313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7b52b045-05d7-4bc4-b2f9-4d8e1d481e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:56.218029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 170/320] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:56.380740 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9588d1-25c8-417b-87a5-e0cfe0adef89 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.381075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9588d1-25c8-417b-87a5-e0cfe0adef89 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.381251 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9588d1-25c8-417b-87a5-e0cfe0adef89 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.382586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 151/321] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:56.387871 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.388157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.388428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.406505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:56.407709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:56.471955 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46e56a-bbdf-48b9-9550-e85e3ab02e91 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.472256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46e56a-bbdf-48b9-9550-e85e3ab02e91 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.472513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46e56a-bbdf-48b9-9550-e85e3ab02e91 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.474008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 152/322] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:04:56.475356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 171/323] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:56.478901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.479188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.479498 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.486778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:56.499053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:56.500349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:56.545765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:56.561990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:56.570880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 153/324] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:56.592072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153779-f45f-4144-973d-d350a6315d01 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:56.626712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:56.627355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.627633 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.627901 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.628979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:56.630434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:56.632201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 172/325] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:04:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:56.637399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:56.737849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01562c-396a-4936-879c-cb4c63ef780b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:56.744064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:56.784248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:56.784865 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.785145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.785426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] PATH_INFO: `/v3/groups` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.786168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:56.787657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:56.804840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 154/326] 10.222.0.18 () {60 vars in 1134 bytes} [Fri Sep 16 19:04:56 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:04:56.843269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:56.873234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33006745-9dd7-4498-86a7-3f86be513d4a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:56.901041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:56.907581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:56.916441 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:56.917087 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:56.917362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:56.917633 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:56.934410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:56.939542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:56.940526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:04:56.941481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 173/327] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:56.946879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:56.962840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35514f-0098-4d11-9d9a-6b6f8536d869 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.000824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.001480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.001755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.002025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.015017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:04:57.022517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:04:57.023508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:04:57.024315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 155/328] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:04:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:04:57.029049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:57.074046 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9731761-6c18-45a8-90ab-4904833448b7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.116619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.117242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.117519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.117791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.118463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:57.118764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:57.121324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:57.130950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 174/329] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:57.137535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:57.153096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5910de2-c2b5-4ad3-aef0-a6ee0ebb8d10 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.169479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:57.193156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.193842 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.194126 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.194398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.195113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:57.195471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:57.196848 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:57.200519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:57.207112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 156/330] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:04:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:57.212882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:57.265764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9242032-1608-4c31-8367-274fbd0801fe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.304005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.305000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.305471 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.305955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] PATH_INFO: `/v3/users/2c1ef9d5602f428484b1b89503f10c3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.307398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] RBAC: Authorizing `identity:get_user(user_id=2c1ef9d5602f428484b1b89503f10c3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:57.309022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:57.311058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 175/331] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:57 2022] GET /identity/v3/users/2c1ef9d5602f428484b1b89503f10c3c => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:57.318425 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:57.358349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71753e7d-8bc2-4e77-83f2-0d02b716203e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.401348 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.402266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.402558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.402838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] PATH_INFO: `/v3/users/6246d854c15a45f6bc7aa0f90873d2be` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.404324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] RBAC: Authorizing `identity:get_user(user_id=6246d854c15a45f6bc7aa0f90873d2be)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:57.406152 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:57.411376 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 157/332] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:04:57 2022] GET /identity/v3/users/6246d854c15a45f6bc7aa0f90873d2be => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:04:57.414681 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:04:57.465584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13b83d-6d7e-4c5d-84f1-f82966a05a9b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.478899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:57.485455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:57.513316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.514223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.514680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.515127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.516108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] RBAC: Adding query filter params (role.id=f2e63848ed864ae5901749a96467215f, user.id=2c1ef9d5602f428484b1b89503f10c3c, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:04:57.516719 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:57.518327 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc0e9-721b-4a48-a40e-54334326c1c6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:57.533137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 176/333] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Sep 16 19:04:57 2022] GET /identity/v3/role_assignments?user.id=2c1ef9d5602f428484b1b89503f10c3c&scope.system=all&role.id=f2e63848ed864ae5901749a96467215f => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:04:57.550751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf78545-a9f1-4c11-8c85-cbbffe575c28 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:04:57.589695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d880994-3827-4ff7-bb0d-602ca8e63403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:04:57.590368 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d880994-3827-4ff7-bb0d-602ca8e63403 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:57.590643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d880994-3827-4ff7-bb0d-602ca8e63403 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:57.590914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d880994-3827-4ff7-bb0d-602ca8e63403 None admin] PATH_INFO: `/v3/system/users/6246d854c15a45f6bc7aa0f90873d2be/roles/7561f8727a4348488f94576d15c61021` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:57.592609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d880994-3827-4ff7-bb0d-602ca8e63403 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6246d854c15a45f6bc7aa0f90873d2be, role_id=7561f8727a4348488f94576d15c61021)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:04:57.594006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d880994-3827-4ff7-bb0d-602ca8e63403 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:04:57.603690 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 158/334] 10.222.0.18 () {58 vars in 1334 bytes} [Fri Sep 16 19:04:57 2022] PUT /identity/v3/system/users/6246d854c15a45f6bc7aa0f90873d2be/roles/7561f8727a4348488f94576d15c61021 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:04:59.528155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66c07-2737-4e3f-a464-12b20945f206 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:59.529248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66c07-2737-4e3f-a464-12b20945f206 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:59.529737 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66c07-2737-4e3f-a464-12b20945f206 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:59.531346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 177/335] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:59.537480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9db87b-1565-44c5-b771-0c917e228bad None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:59.538300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9db87b-1565-44c5-b771-0c917e228bad None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:59.538811 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9db87b-1565-44c5-b771-0c917e228bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:59.561678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1a9db87b-1565-44c5-b771-0c917e228bad None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:59.564090 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1a9db87b-1565-44c5-b771-0c917e228bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:59.658545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 159/336] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:04:59.803352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8ff94-bc35-47fa-8f67-f272a1801ced None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:59.803972 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8ff94-bc35-47fa-8f67-f272a1801ced None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:59.804446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8ff94-bc35-47fa-8f67-f272a1801ced None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:59.806278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 178/337] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:04:59.811355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:04:59.811646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:04:59.811919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:04:59.834273 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:04:59.835558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:04:59.867897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:04:59.919734 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 160/338] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:00.176844 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8ff94-bc35-47fa-8f67-f272a1801ced None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:00.305103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8ff94-bc35-47fa-8f67-f272a1801ced None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:00.345142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:00.346128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.346610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.347099 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.358134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:00.364280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:00.365185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:00.366399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:00.371368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 179/339] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:00.374975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:00.376585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:00.428790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:00.564841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee65149-3197-4754-8ae7-437933d4b16c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:00.619426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cf7bf-781b-4171-a67c-2962ec98ab4b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:00.622026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278624d2-953e-4a9a-b1be-6d3b5a57afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:00.622910 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278624d2-953e-4a9a-b1be-6d3b5a57afb9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.623430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278624d2-953e-4a9a-b1be-6d3b5a57afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.623940 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278624d2-953e-4a9a-b1be-6d3b5a57afb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.637668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278624d2-953e-4a9a-b1be-6d3b5a57afb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-278624d2-953e-4a9a-b1be-6d3b5a57afb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:00.643549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:00.644484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:00.645832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:00.657040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 180/340] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:00.658581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e5896-bff7-49c3-86a8-3286ad489be3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.660420 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e5896-bff7-49c3-86a8-3286ad489be3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.660969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e5896-bff7-49c3-86a8-3286ad489be3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.666646 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 181/341] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:00.668909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e5896-bff7-49c3-86a8-3286ad489be3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:00.694114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:00.695067 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.696742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.697243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.698151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:00.699355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:00.702540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad68989-2d11-40b2-b1f3-6550f42adeb0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:00.724691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 161/342] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:00.726394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.727807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.729160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.751586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:00.753463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:00.796800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:00.871831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:00.882756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 162/343] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:00.884531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:00.890212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e5896-bff7-49c3-86a8-3286ad489be3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:00.956186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:00.956805 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.957086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.957361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.958097 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:00.958403 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:00.960833 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0477cc-7336-4089-acf0-689915a41f16 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:00.973734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 182/344] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:00.975054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e532-0ddc-4f41-813f-759d65927493 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.975367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e532-0ddc-4f41-813f-759d65927493 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.975637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e532-0ddc-4f41-813f-759d65927493 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.977097 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 183/345] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:00.978121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9433-4cff-4383-a52d-4798bdf37c99 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:00.978454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9433-4cff-4383-a52d-4798bdf37c99 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:00.978735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9433-4cff-4383-a52d-4798bdf37c99 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:00.984316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 184/346] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:00.985137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9433-4cff-4383-a52d-4798bdf37c99 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:01.042417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485a572-2a7e-4b00-8bdc-f48aa32f52e5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:01.099361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104040e-3c16-4bca-b9f3-6906e6622d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:01.099982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104040e-3c16-4bca-b9f3-6906e6622d1a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.100260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104040e-3c16-4bca-b9f3-6906e6622d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.100544 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104040e-3c16-4bca-b9f3-6906e6622d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.127270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9433-4cff-4383-a52d-4798bdf37c99 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:01.135733 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:01.177572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19402ad5-8e42-4f61-a6c1-a410cebcab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:01.178180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19402ad5-8e42-4f61-a6c1-a410cebcab95 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.178459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19402ad5-8e42-4f61-a6c1-a410cebcab95 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.178845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19402ad5-8e42-4f61-a6c1-a410cebcab95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.193398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104040e-3c16-4bca-b9f3-6906e6622d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:01.195873 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104040e-3c16-4bca-b9f3-6906e6622d1a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:01.210608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:01.270725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19402ad5-8e42-4f61-a6c1-a410cebcab95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:01.272434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19402ad5-8e42-4f61-a6c1-a410cebcab95 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:01.331153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 163/347] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:01.333118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.333407 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.333680 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.352979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:01.353333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:01.394188 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 185/348] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:01.396231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.396735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.397107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.420508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:01.421989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:01.429385 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:01.448317 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 164/349] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:01.449722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:01.488324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:01.510185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 186/350] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:01.511085 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:01.618524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0564b8d-3e1d-48bd-a258-5d61a9923764 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:01.652018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b17325-3a38-486e-8207-dcb78fec8adf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:01.662201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:01.662962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.663394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.663762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.674225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:01.679254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:01.680342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:01.681164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:01.687620 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 165/351] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:01.688505 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:01.732556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:01.733235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.733513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.733785 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.736337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:01.738050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:01.740094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 187/352] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:01 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:01.740994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:01.748453 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:01.824110 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad753-ec8f-412d-a40b-56d092f9a83b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:01.842998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:01.884985 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:01.886009 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.886499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.886991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.888439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:01.890140 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:01.892058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 166/353] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:01 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:01.893177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:01.906352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2889270-0713-4951-a644-a8903c85a012 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:01.946780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:01.946780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:01.946780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:01.947153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:01.947828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:01.948126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:01.949513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:01.960988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 188/354] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:01.962395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.023563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0d738-2819-4fbe-980f-4b1581abdc91 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.047314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.072675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.073324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.073606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.073881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.081204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.083195 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.084905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 167/355] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:01 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:02.088325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.112540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f841f6-3906-46c6-a4f1-c7c4c2a81a8a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.129115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.154378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.155022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.155317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.155617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.167537 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:02.173449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:02.174408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:02.175223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 189/356] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:02.175946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.203118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c1168-9e3b-45e9-8f21-e2f5b1c9f22b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.247488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.248166 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.248402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.251177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.260115 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:02.265835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:02.266747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:02.267661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 168/357] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:02.268406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.305785 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564ee6-a856-4361-819a-7c73457fd903 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.315058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.345856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.346483 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.346756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.347029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.348714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.350148 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.351906 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 190/358] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:02.356855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.390557 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a106-4910-43b9-ba77-ae3105f6338e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.406152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.433806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.433806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.433806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.433806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.475538 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754c7ad-895b-4381-9ae2-2e0a2b728bce None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.513304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.514769 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.518667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.518667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.518667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.518943 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.519716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:02.520097 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.523333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280868ca-7675-4585-95b6-0544cf541c0f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.529180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 191/359] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:02.529180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0485db-b9dc-4610-89f5-43d3df898098 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.529180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0485db-b9dc-4610-89f5-43d3df898098 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.529501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0485db-b9dc-4610-89f5-43d3df898098 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.531325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 192/360] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:02.534498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0485db-b9dc-4610-89f5-43d3df898098 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.572442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.659554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 169/361] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:02.662471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.690296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0485db-b9dc-4610-89f5-43d3df898098 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.699623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.740690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.741624 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.742103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.742408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.743150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:02.743562 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.745091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.754903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 193/362] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:02.756015 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.795774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205c876-38a6-4d4d-a6ab-adf7f54a5427 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.843457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.844446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.844732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.845005 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.845671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:02.846036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.849144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c53ba-0c3f-47b3-b19d-743460572201 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.857680 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 170/363] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:02.859000 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.859305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.859636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.867920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:02.879964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:02.881334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:02.891267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fe0e9-e935-4114-b806-17bb8c49492a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:02.936906 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:02.938239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:02.938665 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:02.939044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:02.940209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:02.940722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:02.943547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:02.952726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 194/364] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=f2e63848ed864ae5901749a96467215f => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:02.953658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.961114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 171/365] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:02.962442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:02.999676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:03.119829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5a12c-52a0-4b6c-90af-86b195e15553 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.191308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.192351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.192874 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.193422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.194807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:03.196856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:03.198994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 195/366] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:03.200730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.201576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236e51b-ed26-4c1a-9379-87a37a612499 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.241956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.242591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.242874 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.243149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] PATH_INFO: `/v3/users/6246d854c15a45f6bc7aa0f90873d2be` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.244315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] RBAC: Authorizing `identity:get_user(user_id=6246d854c15a45f6bc7aa0f90873d2be)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:03.245751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:03.247674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 172/367] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:05:02 2022] GET /identity/v3/users/6246d854c15a45f6bc7aa0f90873d2be => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:03.248456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.250984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:03.362376 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb53ca-18c8-4b69-b84b-6509c338c7cd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.373248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:03.377277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1efbad-0b37-4f2d-a079-e073c175a08f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.411680 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.413033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.413576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.414103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.415674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:03.416547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.417236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.417585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.417929 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.418160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:03.420086 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 173/368] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:03.421351 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.500372 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:03.502115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:03.513392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:03.555963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca494bc-39be-40e1-8aac-76d9e6515bd5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.601406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.601406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.601406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.601406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.601851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] RBAC: Adding query filter params (role.id=7561f8727a4348488f94576d15c61021, user.id=6246d854c15a45f6bc7aa0f90873d2be, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:03.601938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:03.603743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:03.619718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:03.620962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 174/370] 10.222.0.18 () {56 vars in 1308 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/role_assignments?user.id=6246d854c15a45f6bc7aa0f90873d2be&scope.system=all&role.id=7561f8727a4348488f94576d15c61021 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:03.621333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 196/370] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:03.622037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.622305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.752676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a8557-a5d7-4cfb-9da8-44b5296cdf3f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.785542 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:03.802514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.803178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.803494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.803762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.832743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:03.838170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:03.839073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:03.839895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 175/371] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:03.840560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.881015 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99016e-c774-45d3-b573-53562170bf4c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:03.922883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:03.923516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:03.923792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:03.924116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:03.924933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:03.925290 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:03.926892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:03.933005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 197/372] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=f2e63848ed864ae5901749a96467215f => generated 260 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:03.939800 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:03.976091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc783bb-5f6a-4622-8a03-a3c06a6bdd3a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:04.027586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:04.028155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:04.028433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:04.028703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:04.032747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:04.034475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:04.036888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 176/373] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:04.050137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:04.087859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:04.134472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819818f-de9c-4d2a-b04d-5301d4db6ee9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:04.168399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:04.180583 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:04.181653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:04.181935 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:04.182206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:04.182879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:04.183179 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:04.184793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:04.197276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 198/374] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:04.204228 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:04.253099 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654beaa-f1ec-4783-8147-f18c07425b94 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:04.291194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:04.292243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:04.292798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:04.293415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:04.296340 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:04.298080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:04.300283 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 177/375] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:04 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:04.306145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:04.315758 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c525d9-339e-49a8-b21a-99161c4be86d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:04.385369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:04.416034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:04.416654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:04.416932 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:04.417220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:04.446412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405aa7e-b20e-4a69-b305-ff81fd45da9a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:04.474463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:04.488607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:04.490521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:04.491026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:04.491543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:04.492486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:04.492995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:04.494698 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:04.501251 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 178/376] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:04 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:04.550777 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:04.553933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:04.665064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 199/377] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:04.670904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:04.791967 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e192b-8761-4d68-9fd0-419cc757fd58 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:04.839626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:04.840186 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:04.840468 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:04.840744 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] PATH_INFO: `/v3/projects/3a6c3a4a3b2c47b590674deda9305a68` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:04.846334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] RBAC: Authorizing `identity:get_project(project_id=3a6c3a4a3b2c47b590674deda9305a68)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:04.847827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:04.850471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 179/378] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:04 2022] GET /identity/v3/projects/3a6c3a4a3b2c47b590674deda9305a68 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:04.856040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:04.903321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:04.978274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e9e11-620e-4aa0-b19a-fe7ec95bc91c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:05.023328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa3f2-89c4-49e1-90f9-bd60aff626f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:05.024342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa3f2-89c4-49e1-90f9-bd60aff626f7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:05.024812 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa3f2-89c4-49e1-90f9-bd60aff626f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:05.025274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa3f2-89c4-49e1-90f9-bd60aff626f7 None admin] PATH_INFO: `/v3/groups/f80a45d863704999adc99b6021bf5956` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:05.035994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa3f2-89c4-49e1-90f9-bd60aff626f7 None admin] RBAC: Authorizing `identity:get_group(group_id=f80a45d863704999adc99b6021bf5956)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:05.037784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa3f2-89c4-49e1-90f9-bd60aff626f7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:05.040037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 200/379] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:04 2022] GET /identity/v3/groups/f80a45d863704999adc99b6021bf5956 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:05.045679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:05.139075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:05.196844 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55ef96-8e88-4700-8f5a-ab96a7c334a7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:05.246634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:05.247249 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:05.247550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:05.247823 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:05.248546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] RBAC: Adding query filter params (group.id=f80a45d863704999adc99b6021bf5956, role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=3a6c3a4a3b2c47b590674deda9305a68) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:05.248853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:05.250266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89862a-f1d2-4a8b-82f1-a0805b1ff7ee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:05.256074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 180/380] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:05 2022] GET /identity/v3/role_assignments?group.id=f80a45d863704999adc99b6021bf5956&scope.project.id=3a6c3a4a3b2c47b590674deda9305a68&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:05.449631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745631e-6027-447e-b301-bbee3eae94c8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:05.450302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745631e-6027-447e-b301-bbee3eae94c8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:05.450779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745631e-6027-447e-b301-bbee3eae94c8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:05.452919 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 201/381] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:05.458413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6f903-043a-47d2-aca6-02a5e1ebf287 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:05.458693 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6f903-043a-47d2-aca6-02a5e1ebf287 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:05.459224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6f903-043a-47d2-aca6-02a5e1ebf287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:05.483816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f4e6f903-043a-47d2-aca6-02a5e1ebf287 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:05.485324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f4e6f903-043a-47d2-aca6-02a5e1ebf287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:05.565291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 181/382] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:05.602494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dddfd6-8077-492f-b963-86aacdf735cf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:05.602869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dddfd6-8077-492f-b963-86aacdf735cf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:05.603222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dddfd6-8077-492f-b963-86aacdf735cf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:05.604997 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 202/383] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:05.609978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:05.610276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:05.610555 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:05.636286 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:05.637523 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:05.695660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:05.730031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 182/384] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:05.899522 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dddfd6-8077-492f-b963-86aacdf735cf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.008894 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:06.024929 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dddfd6-8077-492f-b963-86aacdf735cf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.063322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.064029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.064316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.064590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.075939 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:06.081499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:06.082382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:06.083272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:06.087376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 203/385] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:06.089153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.136802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.247420 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91853d2c-0a98-4901-bb9c-ac55cc1d627d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.288318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.288873 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.289374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.289648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.290393 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:06.290699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:06.292631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:06.302267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 183/386] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:06.307784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.328408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9fea4-8a85-4be3-a28e-09fb7e862d48 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.383213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:06.394023 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.394668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.395088 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.395395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.417293 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:06.425863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:06.426892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:06.427735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 204/387] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:06.437250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.455960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:06.485835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03670ee6-57ff-4f55-819d-b3fe27dc6273 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.549693 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.550335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.550615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.550887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.554121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:06.555650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:06.563113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 184/388] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:06.572579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.614764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574adb-e214-47c9-a571-f5b2607d6ab7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.674426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.675535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.676141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.678316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.679854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:06.680962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:06.682680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:06.699699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 205/389] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:06.704054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.752181 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5575e5c-1252-4c11-b5c7-7059f77aaafc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.769864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:06.816308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.817989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.820607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.821092 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.823718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:06.826016 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:06.830100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 185/390] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:06.835103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:06.836447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:06.893318 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da199-68c7-4c07-a30d-0bc5b3760f74 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:06.967419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032811-9d90-43a9-819d-cf3e68c178d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:06.968072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032811-9d90-43a9-819d-cf3e68c178d1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.968554 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032811-9d90-43a9-819d-cf3e68c178d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.968836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032811-9d90-43a9-819d-cf3e68c178d1 None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.970579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032811-9d90-43a9-819d-cf3e68c178d1 None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:06.972390 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032811-9d90-43a9-819d-cf3e68c178d1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:06.975960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 206/391] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:06.977161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e653af-0847-41a5-bea8-a3e75ce55dd3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:06.977445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e653af-0847-41a5-bea8-a3e75ce55dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:06.977722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e653af-0847-41a5-bea8-a3e75ce55dd3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:06.979084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 207/392] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:06.984309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e653af-0847-41a5-bea8-a3e75ce55dd3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:07.012283 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aeeca-a1ea-4d42-9ece-2bd84158b895 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:07.048354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7632-7bd0-46b6-ac6b-93277c831fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:07.049415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7632-7bd0-46b6-ac6b-93277c831fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.049905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7632-7bd0-46b6-ac6b-93277c831fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.050377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7632-7bd0-46b6-ac6b-93277c831fa3 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.052896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7632-7bd0-46b6-ac6b-93277c831fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac2ffc24a14d4dc1a43619feba9ae47b, group_id=b54a59ae44fd4b3e8f60bf131a645082, role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:07.054558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7632-7bd0-46b6-ac6b-93277c831fa3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:07.079511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 186/393] 10.222.0.18 () {58 vars in 1442 bytes} [Fri Sep 16 19:05:06 2022] PUT /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:07.079511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4272ff-f62a-4414-881f-8789e547b924 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.079511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4272ff-f62a-4414-881f-8789e547b924 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.079511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4272ff-f62a-4414-881f-8789e547b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.106034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ac4272ff-f62a-4414-881f-8789e547b924 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:07.108080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ac4272ff-f62a-4414-881f-8789e547b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:07.116788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e653af-0847-41a5-bea8-a3e75ce55dd3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:07.147988 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:07.157988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584e4de-9092-4158-ac3a-e1762e2e921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:07.158401 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584e4de-9092-4158-ac3a-e1762e2e921f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.158683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584e4de-9092-4158-ac3a-e1762e2e921f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.158962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584e4de-9092-4158-ac3a-e1762e2e921f None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.161597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584e4de-9092-4158-ac3a-e1762e2e921f None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:07.163074 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584e4de-9092-4158-ac3a-e1762e2e921f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:07.170005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 208/394] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:06 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:07.171711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ede00-07ec-4dbc-9022-e2393f52706c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.171864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ede00-07ec-4dbc-9022-e2393f52706c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.172090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ede00-07ec-4dbc-9022-e2393f52706c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.173760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 209/395] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:07.174514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ede00-07ec-4dbc-9022-e2393f52706c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:07.186843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:07.213431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 187/396] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:07.213431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.213431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.213431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.247957 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:07.249191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:07.325213 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ede00-07ec-4dbc-9022-e2393f52706c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:07.349224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 188/397] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:07.393387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:07.394060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.394446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.394723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.400560 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac2ffc24a14d4dc1a43619feba9ae47b, group_id=b54a59ae44fd4b3e8f60bf131a645082, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:07.401992 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:07.419688 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 210/398] 10.222.0.18 () {58 vars in 1442 bytes} [Fri Sep 16 19:05:07 2022] PUT /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:07.526655 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:07.634154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:07.638784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00699dd-1a42-4aed-83cb-1daf379bfad9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:07.677016 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:07.678042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.678524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.678995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.690076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:07.695924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:07.696887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:07.697810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:07.700795 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 189/399] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:07.706859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:07.715473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:07.828823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:07.876281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026d70-3aa6-4443-a6c1-004bc44b7f38 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:07.927140 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:07.927718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.927997 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.928270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:07.928957 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:07.929261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:07.930662 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:07.934487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45943f5-250e-459c-a55a-f3894b7bf561 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:07.953907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 211/400] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:07.961196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:07.994215 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd965f-6d8a-454b-a206-c9c1fcf10c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:07.994857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd965f-6d8a-454b-a206-c9c1fcf10c55 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:07.995136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd965f-6d8a-454b-a206-c9c1fcf10c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:07.995442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd965f-6d8a-454b-a206-c9c1fcf10c55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.012267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd965f-6d8a-454b-a206-c9c1fcf10c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8dfd965f-6d8a-454b-a206-c9c1fcf10c55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:08.017203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:08.018088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:08.018906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:08.026079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 190/401] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:08.027599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f790063-3841-4187-8e52-479232e01fda None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.027850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f790063-3841-4187-8e52-479232e01fda None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.028222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f790063-3841-4187-8e52-479232e01fda None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.029514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 191/402] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:08.030192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f790063-3841-4187-8e52-479232e01fda None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.055061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.110657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39006fc6-050d-4c9d-9541-b67dd312740e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.147446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.155216 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e1554-8000-4f97-944b-3a0f655d0dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.155923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e1554-8000-4f97-944b-3a0f655d0dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.156203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e1554-8000-4f97-944b-3a0f655d0dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.156577 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e1554-8000-4f97-944b-3a0f655d0dc6 None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.157792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e1554-8000-4f97-944b-3a0f655d0dc6 None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.159781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e1554-8000-4f97-944b-3a0f655d0dc6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.162645 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 212/403] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:07 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:08.163886 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.164173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.164459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.165598 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f790063-3841-4187-8e52-479232e01fda None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.184426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:08.185818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:08.209934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.210653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.210936 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.211207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.211908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:08.212209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.213683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.222573 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 192/404] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:08.223461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.275735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 213/405] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:08.276760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.318763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.363443 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9d00-7081-47de-855a-a1121338c2c1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.386462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c330a4-a1f6-4555-92b7-b81a56136a1c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.415666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.424356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.425225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.425644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.426057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.427596 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.429386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.431419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 214/406] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:08.437497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.445318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.445963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.446248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.446521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.452135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.453677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.455465 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 193/407] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:08.461645 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.544728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68df497-2732-4440-92ce-d78771158d29 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.586269 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.587030 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.587328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.587602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.588803 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.590477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.593025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 215/408] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:08.593783 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.627917 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f7e19-e31e-4503-9270-d963070fafe8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.668887 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.673117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.673948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.674226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.674496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.681124 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] RBAC: Authorizing `identity:create_grant(project_id=015ebad4cd344e7f8b6506b57b7a7b90, group_id=b54a59ae44fd4b3e8f60bf131a645082, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.682638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.703712 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 194/409] 10.222.0.18 () {58 vars in 1442 bytes} [Fri Sep 16 19:05:08 2022] PUT /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:05:08.704835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.720509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.724281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf881-e9aa-4bb9-a105-61683b67adf0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.769922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.771079 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.771383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.771651 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.783574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:08.789215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:08.790095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:08.790987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 216/410] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:08.796637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:08.861350 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dbf4d-3607-4e68-983a-8e3387f1834b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.915902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de121-899d-4a4a-9450-dadf7ba9f192 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:08.920625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.921618 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.922172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.922679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.929359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] RBAC: Authorizing `identity:create_grant(project_id=015ebad4cd344e7f8b6506b57b7a7b90, group_id=b54a59ae44fd4b3e8f60bf131a645082, role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.931164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.946427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:08.956173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:08.957064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:08.960367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:08.960367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:08.960367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:08.960367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:08.961359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:08.965188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 195/411] 10.222.0.18 () {58 vars in 1442 bytes} [Fri Sep 16 19:05:08 2022] PUT /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90/groups/b54a59ae44fd4b3e8f60bf131a645082/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:08.975947 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 217/412] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:08.978168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:09.068384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:09.114771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff41dd4-b63c-4691-a742-25d86414479c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:09.164085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:09.164734 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:09.165019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:09.165297 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] PATH_INFO: `/v3/groups/f80a45d863704999adc99b6021bf5956` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:09.169414 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] RBAC: Authorizing `identity:get_group(group_id=f80a45d863704999adc99b6021bf5956)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:09.170891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:09.172998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 196/413] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:08 2022] GET /identity/v3/groups/f80a45d863704999adc99b6021bf5956 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:09.178559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:09.297656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82430ec5-4be3-4319-89e9-46f074fd4808 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:09.336360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e68d5-7573-4bec-86a5-d2b0832bae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:09.337017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e68d5-7573-4bec-86a5-d2b0832bae5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:09.337297 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e68d5-7573-4bec-86a5-d2b0832bae5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:09.337634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e68d5-7573-4bec-86a5-d2b0832bae5b None admin] PATH_INFO: `/v3/projects/3a6c3a4a3b2c47b590674deda9305a68` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:09.338879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e68d5-7573-4bec-86a5-d2b0832bae5b None admin] RBAC: Authorizing `identity:get_project(project_id=3a6c3a4a3b2c47b590674deda9305a68)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:09.340399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e68d5-7573-4bec-86a5-d2b0832bae5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:09.342384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 218/414] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:09 2022] GET /identity/v3/projects/3a6c3a4a3b2c47b590674deda9305a68 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:09.347993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:09.473547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490afb1-329a-4fc0-8a80-969d171a7f53 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:09.486746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f445ab-c93c-4f44-9942-5c7874123d9f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:09.487277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f445ab-c93c-4f44-9942-5c7874123d9f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:09.487586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f445ab-c93c-4f44-9942-5c7874123d9f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:09.489596 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 219/415] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:05:09.494871 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:09.495434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:09.495929 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:09.519643 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:09.522726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:09.532771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:09.533785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:09.534278 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:09.534743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] PATH_INFO: `/v3/projects/3a6c3a4a3b2c47b590674deda9305a68/groups/f80a45d863704999adc99b6021bf5956/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:09.539301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] RBAC: Authorizing `identity:create_grant(project_id=3a6c3a4a3b2c47b590674deda9305a68, group_id=f80a45d863704999adc99b6021bf5956, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:09.540966 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:09.546653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:09.562020 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 197/416] 10.222.0.18 () {58 vars in 1442 bytes} [Fri Sep 16 19:05:09 2022] PUT /identity/v3/projects/3a6c3a4a3b2c47b590674deda9305a68/groups/f80a45d863704999adc99b6021bf5956/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:09.599707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:09.632176 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 220/417] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:10.001786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:10.138339 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362a772-a656-4616-a574-40319dc8dd2e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:10.175869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c1746-1ed6-4471-b469-2902338dd3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:10.177163 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c1746-1ed6-4471-b469-2902338dd3ba None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:10.177400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c1746-1ed6-4471-b469-2902338dd3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:10.177637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c1746-1ed6-4471-b469-2902338dd3ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:10.189165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c1746-1ed6-4471-b469-2902338dd3ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ca1c1746-1ed6-4471-b469-2902338dd3ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:10.195322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:10.196192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:10.196995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 198/418] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:10.202587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:10.302283 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de0e4a-f6d8-45d4-90c9-2d649882a382 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:10.302970 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de0e4a-f6d8-45d4-90c9-2d649882a382 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:10.303583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de0e4a-f6d8-45d4-90c9-2d649882a382 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:10.306284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 199/419] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:10.309890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94aa6f-0f16-4981-b7a9-e6fb2bdcfedd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:10.313660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:10.314201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:10.314677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:10.334137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:10.335800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:10.359514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:10.360128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:10.360402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:10.360669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:10.361346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:10.361780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:10.364630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:10.374300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 221/420] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:10.383695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:10.395747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:10.410797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 200/421] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:10.455836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:10.520828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742395a-b25a-4110-9eec-7f110725e656 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:10.554387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:10.555406 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:10.555688 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:10.555957 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:10.641936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:10.643386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:10.702539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:10.733608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:10.795344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 222/422] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:10.801891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:10.906453 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb375c5-235f-4c4b-8547-101b97d45cdf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:10.945625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:10.946623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:10.947102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:10.947631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:10.960975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:10.966918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:10.967837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:10.968627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:10.971677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 201/423] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:10.979135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:10.990062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bc5fa-d922-4305-ac2b-1ff5bb9de4cf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:11.044212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:11.048308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:11.049009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.049289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.049562 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.050876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:11.052911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:11.054623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 223/424] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:10 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:11.060464 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:11.094502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:11.114552 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffa34d-efc4-4956-8008-2f6cbf42c2f7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:11.201501 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:11.202103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.202331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.202540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.203013 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da5f2f-5a7f-442c-bae3-955ed3a0665e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:11.203406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:11.203561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:11.204879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:11.222611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 202/425] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:11.228294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:11.245881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:11.246523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.246801 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.247072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.248813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:11.249685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:11.251578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 224/426] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:11 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:11.258006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:11.401325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9e602-0433-4139-bcc8-1f5c72402634 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:11.425926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:11.427029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:11.458989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9077cf-d0af-49ed-875c-3efeeaff462d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:11.475797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:11.476419 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.476697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.476968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.508829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:11.509437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.509712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.509981 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.510703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:11.511011 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:11.512461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0e8c-33e4-4428-81ac-1f5c50485659 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:11.522319 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 225/427] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:11 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 611 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:11.523608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212566cd-5edc-47c2-90ea-ee771fff9a09 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.523845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212566cd-5edc-47c2-90ea-ee771fff9a09 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.524122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212566cd-5edc-47c2-90ea-ee771fff9a09 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.525509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 226/428] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:11.531105 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.531422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.531695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.550726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:11.552035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:11.570821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:11.572718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:11.627450 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 227/429] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:11.688489 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 203/430] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 19:05:11.696749 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:11.801918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:11.826671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae290b2-e336-4cf3-be1a-1b3607449f7b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:11.862614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab59e-edac-409b-9576-a7b92965c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:11.863246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab59e-edac-409b-9576-a7b92965c9ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.863548 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab59e-edac-409b-9576-a7b92965c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.864343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab59e-edac-409b-9576-a7b92965c9ef None admin] PATH_INFO: `/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.865478 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab59e-edac-409b-9576-a7b92965c9ef None admin] RBAC: Authorizing `identity:get_project(project_id=ac2ffc24a14d4dc1a43619feba9ae47b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:11.866920 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab59e-edac-409b-9576-a7b92965c9ef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:11.868938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 228/431] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:11 2022] GET /identity/v3/projects/ac2ffc24a14d4dc1a43619feba9ae47b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:11.874164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:11.889796 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:11.908152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506186a-f32b-492b-b52c-57da562d691d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.908447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506186a-f32b-492b-b52c-57da562d691d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.908706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506186a-f32b-492b-b52c-57da562d691d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.910196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 229/432] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:11.923197 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:11.923530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:11.923819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:11.946922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:11.948656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:12.019662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8f037-c28f-4b04-a632-ccb731811125 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.030618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 230/433] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:12.031945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.076092 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.076992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.077531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.078001 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.079527 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:12.081294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:12.089154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 204/434] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:11 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:12.090415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.118646 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.145367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4a0c3-db0f-456b-ab5b-24772c1bb9ab None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.181764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.183786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.184428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.185022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.185117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.201419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:12.207965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:12.209278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 19:05:12.210064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:12.210673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21178e1e-b8cf-4640-93db-3bb41092ee94 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.217804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 231/435] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:12.218969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.255136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.256050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.256601 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.257301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.259088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:12.259603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:12.261183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:12.269089 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 205/436] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=ac2ffc24a14d4dc1a43619feba9ae47b&role.id=f2e63848ed864ae5901749a96467215f => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:12.327658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.340147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c0ca4-bfae-4d51-8b05-63b6bbc91f97 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.384961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.385586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.385866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.386137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.386828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:12.387129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:12.388603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:12.390573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.402796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 232/437] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:12.412626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.488424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22b47f-17a6-4b92-b004-2237abfff5ec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.534726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385f61e-168f-437a-94fa-b4e4d3ff21d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.535393 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385f61e-168f-437a-94fa-b4e4d3ff21d3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.536298 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385f61e-168f-437a-94fa-b4e4d3ff21d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.536579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385f61e-168f-437a-94fa-b4e4d3ff21d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.543057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a97c-0f07-4479-9e18-f4f88b7b9100 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.548485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.553288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385f61e-168f-437a-94fa-b4e4d3ff21d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d385f61e-168f-437a-94fa-b4e4d3ff21d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:12.558640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:12.559578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:12.560402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:12.561019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 206/438] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:12.562267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856f7a0-48af-41b5-b516-b39dd4169819 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.562549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856f7a0-48af-41b5-b516-b39dd4169819 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.563419 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856f7a0-48af-41b5-b516-b39dd4169819 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.564333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 207/439] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:12.566568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856f7a0-48af-41b5-b516-b39dd4169819 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.584109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.585089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.585773 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.586056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.657157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.665122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:12.666621 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:12.683957 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856f7a0-48af-41b5-b516-b39dd4169819 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.728079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.728705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.728980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.729250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.730003 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:12.730301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:12.731941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2730e0b-5320-43eb-aab3-0fb99751b194 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:12.742917 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 208/440] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:12.744329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.744619 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.744893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.762800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:12.764020 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:12.783001 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 233/441] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:12.783937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.828257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.845237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 209/442] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:12.845815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:12.899208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:12.906493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b378667-fce4-4116-b5b1-df9e6c2103bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.941800 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.942936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.943212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.943508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.955167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa1fa4-4096-49a9-a7d8-44ad7628fd08 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:12.991260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:12.992291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:12.992571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:12.993114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:12.994446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:12.996266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:12.998455 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 210/443] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.003653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.016090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.017399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.075360 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.119256 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.120634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359c513-85c0-42f4-a26a-f149dafdc684 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.127152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 234/444] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:13.128060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.158443 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.159073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.159378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.159652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.161286 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.162717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.164295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 211/445] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:13.165044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.228617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5250e-005a-436f-b2d8-d56e038989de None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.261795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.262024 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934da1e-f567-4275-bfea-e92d984a2baa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.262561 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.262930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.263273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.273893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:13.278799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:13.279765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:13.280556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 235/446] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:13.281201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.294590 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.302986 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.303636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.303915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.304188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] PATH_INFO: `/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.305297 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] RBAC: Authorizing `identity:get_project(project_id=015ebad4cd344e7f8b6506b57b7a7b90)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.306717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.312775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 212/447] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/projects/015ebad4cd344e7f8b6506b57b7a7b90 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.312775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.337298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.380781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a92c63-ae5d-4b07-b293-b2bd3633b13f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.408632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe2cd-3a5f-45e1-bd88-9adc44aa1b1e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.413724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.414320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.414601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.414874 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.415624 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=f2e63848ed864ae5901749a96467215f, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:13.415932 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.417358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.422924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 236/448] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=f2e63848ed864ae5901749a96467215f => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.423725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.444600 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.445223 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.445500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.445770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.446450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:13.446747 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.448204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.463010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 213/449] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.464213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.532972 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.547715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543625af-2ed2-46a5-ad21-0f746bdc90b1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.570419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.582363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3fd20-0189-4d07-aadd-86c4f7b2cf2a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.587123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.587793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.588076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.588351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] PATH_INFO: `/v3/groups/b54a59ae44fd4b3e8f60bf131a645082` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.589481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] RBAC: Authorizing `identity:get_group(group_id=b54a59ae44fd4b3e8f60bf131a645082)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.591326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.592543 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 237/450] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/groups/b54a59ae44fd4b3e8f60bf131a645082 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.601217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.623546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.624163 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.624447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.624725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.702204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.704966 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.720564 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2b47-8d75-4add-968b-72451a3c1e5c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.761652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.762265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.762547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.762816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.763570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] RBAC: Adding query filter params (group.id=b54a59ae44fd4b3e8f60bf131a645082, role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:13.763879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.764879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.765304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.772231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 238/451] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/role_assignments?group.id=b54a59ae44fd4b3e8f60bf131a645082&scope.project.id=015ebad4cd344e7f8b6506b57b7a7b90&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.821775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 214/452] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 19:05:13.827943 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:13.884259 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:13.934559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cda1e6-d9d5-410e-a66b-e1032d875a11 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:13.969192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:13.969749 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:13.970034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:13.970311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] PATH_INFO: `/v3/projects/3a6c3a4a3b2c47b590674deda9305a68` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:13.971468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=3a6c3a4a3b2c47b590674deda9305a68)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:13.972896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:13.974553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 239/453] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/projects/3a6c3a4a3b2c47b590674deda9305a68 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:13.980200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:14.081843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391d76e-3d20-479e-8734-f5433b69ec04 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:14.117162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f33a0-233c-4673-b6ec-cf36a4bec099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:14.117795 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f33a0-233c-4673-b6ec-cf36a4bec099 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:14.118074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f33a0-233c-4673-b6ec-cf36a4bec099 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:14.118353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f33a0-233c-4673-b6ec-cf36a4bec099 None admin] PATH_INFO: `/v3/groups/f80a45d863704999adc99b6021bf5956` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:14.119505 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f33a0-233c-4673-b6ec-cf36a4bec099 None admin] RBAC: Authorizing `identity:get_group(group_id=f80a45d863704999adc99b6021bf5956)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:14.120956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f33a0-233c-4673-b6ec-cf36a4bec099 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:14.122513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 215/454] 10.222.0.18 () {56 vars in 1180 bytes} [Fri Sep 16 19:05:13 2022] GET /identity/v3/groups/f80a45d863704999adc99b6021bf5956 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:14.128420 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:14.224236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9ced3-b3a3-41da-9864-ca106497b1f1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:14.243253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:14.257142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:14.257736 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:14.258017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:14.258287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:14.258999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] RBAC: Adding query filter params (group.id=f80a45d863704999adc99b6021bf5956, role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=3a6c3a4a3b2c47b590674deda9305a68) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:14.259327 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:14.260734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c828f-c840-4a6d-8ae4-bedf913e81be None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:14.266250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 240/455] 10.222.0.18 () {56 vars in 1376 bytes} [Fri Sep 16 19:05:14 2022] GET /identity/v3/role_assignments?group.id=f80a45d863704999adc99b6021bf5956&scope.project.id=3a6c3a4a3b2c47b590674deda9305a68&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:16.813321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a06d2-7b48-4c1a-844b-0f469ea80143 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:16.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a06d2-7b48-4c1a-844b-0f469ea80143 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:16.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a06d2-7b48-4c1a-844b-0f469ea80143 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:16.815463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 216/456] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:16.821279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad9cb1-c213-4492-9e2c-485bdf077dc0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:16.822081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad9cb1-c213-4492-9e2c-485bdf077dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:16.822551 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad9cb1-c213-4492-9e2c-485bdf077dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:16.843560 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f5ad9cb1-c213-4492-9e2c-485bdf077dc0 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:16.845108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f5ad9cb1-c213-4492-9e2c-485bdf077dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:16.946991 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 241/457] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:16.996733 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55eb63-0f3a-40f9-a3af-f3ca9bc1009f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:16.997375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55eb63-0f3a-40f9-a3af-f3ca9bc1009f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:16.997837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55eb63-0f3a-40f9-a3af-f3ca9bc1009f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:16.999631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 217/458] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:17.004730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.005020 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.005298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.029225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:17.030429 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:17.129809 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 242/459] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:17.328596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55eb63-0f3a-40f9-a3af-f3ca9bc1009f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:17.365671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:17.463081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:17.472341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55eb63-0f3a-40f9-a3af-f3ca9bc1009f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:17.508813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:17.509471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.509755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.510027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.521235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:17.522726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fe23f3c0-7f72-4065-b032-18ad4abe05a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:17.540648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:17.541606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:17.542460 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:17.542460 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:17.542460 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:17.542460 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 218/460] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:17.543473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74459b-c5ac-4efd-b78c-046f9dd55b69 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.543758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74459b-c5ac-4efd-b78c-046f9dd55b69 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.544031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74459b-c5ac-4efd-b78c-046f9dd55b69 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.545509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 219/461] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:17.551340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:17.553171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.553738 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.553738 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.573790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:17.575258 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:17.584565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76159421-f4ee-4b43-87f2-3881d6634974 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:17.629725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:17.630385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.630667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.630967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.642806 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:17.644308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:17.649300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 220/462] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:17.650463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:17.661388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-42c4aa79-7957-4333-ad29-59d9f040de05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:17.668491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:17.669375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:17.670273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:17.670273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:17.670273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:17.672838 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 243/463] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:17.674284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603344-f953-49a4-b20a-a0b9e13f4aab None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.674806 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603344-f953-49a4-b20a-a0b9e13f4aab None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.675265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603344-f953-49a4-b20a-a0b9e13f4aab None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.681682 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 244/464] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:17.681682 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603344-f953-49a4-b20a-a0b9e13f4aab None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:17.782249 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd6f8a-1aa3-4ec1-9d12-436cfd9c5e5e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:17.794961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:17.807197 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603344-f953-49a4-b20a-a0b9e13f4aab None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:17.818359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:17.818962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.819438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.819851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.820738 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:17.821200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:17.822830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974187d-01e8-4a28-b240-1f60c64f9848 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:17.846459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 221/465] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:17.847902 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.848384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.848706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:17.849200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:17.850528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:17.851051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:17.852288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:17.852785 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:17.854380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:17.868930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:17.870301 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 245/466] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:17.871359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:17.874551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:17.943574 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:17.949304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 222/467] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:17.950094 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:17.994442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9189ec-1840-49d1-8dc7-5ffb9c8430e3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.030923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.031907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.032381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.032946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.033686 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.035336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.066113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e2e9a-71a8-48a7-8fb3-8748a711bc13 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.067433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 246/468] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:05:17 2022] POST /identity/v3/users => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:18.068456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:18.105552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:18.145771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.146942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.148482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.148938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.150987 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.155682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.201057 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 223/469] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:05:17 2022] POST /identity/v3/users => generated 248 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:18.258687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:18.277325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c05ef-a926-431b-ba47-7f4091291675 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.320468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.321549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.321920 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.322290 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.334784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:18.342617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.344426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:18.367161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:18.368266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:18.369377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:18.369377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:18.369377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:18.369377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 247/470] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:18.374492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:18.403131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184bb29-425c-43ac-81ee-6aa0751014b6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.451946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.452836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.453262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.453533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.456712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:18.467516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.468995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:18.485789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:18.486725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:18.488334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:18.488334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:18.488334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:18.488334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 224/471] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:18.494420 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:18.505225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061512-4d0b-4393-a440-0386924ea470 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.546106 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.546947 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.547354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.547730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.548515 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:18.548922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.550466 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.569463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 248/472] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:18.576401 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:18.616244 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bd6b0-266e-4e61-9ce4-47fbdf0ec471 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.623926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:18.657997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.658718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.659058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.659409 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.660134 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:18.660501 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.662113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.677525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 225/473] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:18.682812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:18.708599 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887f60b-51a0-4415-ac2b-6ecb4edcce16 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.736358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:18.751764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5f538-9b1e-44a7-a05b-9258f3d22468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.752414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f538-9b1e-44a7-a05b-9258f3d22468 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.753109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f538-9b1e-44a7-a05b-9258f3d22468 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.753391 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5f538-9b1e-44a7-a05b-9258f3d22468 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.754323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5f538-9b1e-44a7-a05b-9258f3d22468 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.755950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5f538-9b1e-44a7-a05b-9258f3d22468 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.781596 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 249/474] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:05:18 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:05:18.783011 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10fb3f-534f-473d-8074-720e1a19f10f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.783384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10fb3f-534f-473d-8074-720e1a19f10f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.783707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10fb3f-534f-473d-8074-720e1a19f10f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.790258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 250/475] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:18.791942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.792329 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.792935 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.802706 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0c28c-961d-4a6b-a716-817c5221f270 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:18.817760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:18.819375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:18.843856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:18.845091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:18.845517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:18.845934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:18.846839 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:18.848642 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:18.878392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 226/476] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:05:18 2022] POST /identity/v3/users => generated 249 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:18.904765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 251/477] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:19.184668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:19.211821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:19.294784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df48ba-ccb2-4fc5-9972-98483b45a3af None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:19.334201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:19.334815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:19.335093 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:19.335451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:19.336135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:19.337508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:19.350263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 19:05:19.350263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 16 19:05:19.350263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [parameters: {'id': '133e88ac605f4473b4ccb07cbeb3508a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 16 19:05:19.350263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 16 19:05:19.374193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return ref.to_dict() Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 19:05:19.375129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 19:05:19.376066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 19:05:19.376938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 19:05:19.377818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 19:05:19.378623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'id': '133e88ac605f4473b4ccb07cbeb3508a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:19.379576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 16 19:05:19.380365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:19.381269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 227/478] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Sep 16 19:05:19 2022] POST /identity/v3/roles => generated 144 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:19.392258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:19.523766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08227b4-2b95-4c9a-a092-b57c59eddf33 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:19.542259 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:19.576030 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ebda8-cf6b-4437-993c-e8f8faaefd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:19.577025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ebda8-cf6b-4437-993c-e8f8faaefd9c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:19.577522 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ebda8-cf6b-4437-993c-e8f8faaefd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:19.578004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ebda8-cf6b-4437-993c-e8f8faaefd9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:19.596113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ebda8-cf6b-4437-993c-e8f8faaefd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a47ebda8-cf6b-4437-993c-e8f8faaefd9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:19.601784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:19.602637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 19:05:19.603480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:19.605684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 252/479] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:19.611614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:19.710119 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:19.728900 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffec7d2-be77-4711-b33f-00666ad328b3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:19.775128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:19.775701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:19.779382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:19.779673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:19.780365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:19.780677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:19.782051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50bcad-2504-49ce-8989-4bd8daa10c60 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:19.791405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 228/480] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:20.751216 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663ea1-f786-4c33-b447-cfe4f7570ef5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:20.751770 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663ea1-f786-4c33-b447-cfe4f7570ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:20.751770 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e663ea1-f786-4c33-b447-cfe4f7570ef5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:20.753383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 253/481] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:20.758695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5d69-878f-4739-9268-bf798c2b6139 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:20.759193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5d69-878f-4739-9268-bf798c2b6139 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:20.759474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5d69-878f-4739-9268-bf798c2b6139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:20.784267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-523e5d69-878f-4739-9268-bf798c2b6139 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:20.785466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-523e5d69-878f-4739-9268-bf798c2b6139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:20.877993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 229/482] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:20.985915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba6be-a16b-47a3-816e-4449812c92ac None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:20.986230 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba6be-a16b-47a3-816e-4449812c92ac None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:20.986479 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbba6be-a16b-47a3-816e-4449812c92ac None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:20.988088 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 254/483] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:20.993869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:20.994165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:20.994445 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:21.020876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:21.022002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:21.114953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 230/484] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:21.314662 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbba6be-a16b-47a3-816e-4449812c92ac None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:21.445077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbba6be-a16b-47a3-816e-4449812c92ac None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:21.496756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:21.497804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:21.498294 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:21.498753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:21.512832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:21.513965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:21.519746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:21.521157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:21.521946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:21.524577 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 255/485] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:21.531202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:21.531639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:21.563673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:21.635151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57db79-f858-4df9-be6c-18b7ba2a1bac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:21.680114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:21.680723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:21.681000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:21.681273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:21.681936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:21.682238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:21.683634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:21.703265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 256/486] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:21.705546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:21.711277 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eb77a-4370-488c-9db2-b0d58d24e010 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:21.787811 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:21.788457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:21.788737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:21.788991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:21.812253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:21.818026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:21.818904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:21.819732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 231/487] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:21.834696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:21.836278 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:21.853100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b9e45-3b42-4b02-8f43-764dcea4fe26 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:21.889719 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:21.891022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:21.891022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:21.891022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:21.902137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:21.903797 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fdcd39c7-e437-496b-892a-0b10b78e428d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:21.919485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:21.920572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:21.921623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:21.921623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:21.921623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:21.927412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 257/488] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:21.928853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90201a8b-9a34-4d41-80d2-9251d4a22e0c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:21.929236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90201a8b-9a34-4d41-80d2-9251d4a22e0c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:21.929612 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90201a8b-9a34-4d41-80d2-9251d4a22e0c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:21.935260 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 258/489] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:21.935260 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90201a8b-9a34-4d41-80d2-9251d4a22e0c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:21.936579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:21.980014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7faa6-da21-4310-8db0-1c7c04e21558 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.018080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.018684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.018960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.019229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.020427 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:22.020427 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.021653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5c7b7-f1a0-4cb9-8c43-1ff3bf354db4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.034568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 232/490] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:22.034568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.034568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.034568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.050682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:22.052057 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:22.066635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90201a8b-9a34-4d41-80d2-9251d4a22e0c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.123343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.124270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.124676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.125138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.126431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:22.127252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.128942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f1d56-0421-42f9-a1c6-d0946f2a47dd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.147382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 233/491] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:22.147382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.153477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 259/492] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:22.154792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58eb6be-d62e-41dc-bc11-3cce9368c942 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.155176 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58eb6be-d62e-41dc-bc11-3cce9368c942 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.155473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58eb6be-d62e-41dc-bc11-3cce9368c942 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.156662 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 260/493] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:22.161945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58eb6be-d62e-41dc-bc11-3cce9368c942 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.166557 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.210875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.252769 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b9776-c253-4ea5-863e-a6a8a1c2d533 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.277412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58eb6be-d62e-41dc-bc11-3cce9368c942 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.288095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.288808 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.289156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.289494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.299852 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.301897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.314077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.314699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.314977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.315310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-796cb41c-f052-49df-88d6-cc00af61e3c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:22.316956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:22.317810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:22.318666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:22.318666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:22.318666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:22.322670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 234/494] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:22.323027 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.323959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.324301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.324535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.324704 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:22.339107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:22.341233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 261/495] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:22.343570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.347892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:22.349183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:22.420875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 235/496] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:22.421802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.430309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.471740 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.487926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb6327-4ce3-41ae-9d7b-733aed60c2f2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.532718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ba8c-36d0-45e3-afea-b3aca8c23030 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.533589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.534607 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.535718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.536003 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.536685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:22.536994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.538378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.550534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 262/497] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:22.555789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.572748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.573359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.573640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.573920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.574651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:22.574962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.576451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee7331-a471-454f-9ba8-2e4e6935a27a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.615120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 236/498] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:22.616661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6dd2b-926a-43ba-bbe9-370e182318b6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.616947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6dd2b-926a-43ba-bbe9-370e182318b6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.617236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6dd2b-926a-43ba-bbe9-370e182318b6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.623182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 237/499] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:22.623182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6dd2b-926a-43ba-bbe9-370e182318b6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.666820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b4438-db3c-4bab-9101-a556d0b6370d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.694532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.707038 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.708252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.708529 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.708804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.718665 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:22.723521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:22.724545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:22.725366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 263/500] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:22.726016 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.726210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.731222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6dd2b-926a-43ba-bbe9-370e182318b6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.765208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.765822 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.766098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.766363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.774568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.776049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-029fd97a-e5d4-4b16-9401-59ac61949e98 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:22.788430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:22.789728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 238/501] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:22.790612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.791221 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.791221 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.807993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:22.809165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:22.830782 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232513b-92e2-497f-bc92-d192b18bab75 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:22.864558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:22.865135 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:22.865409 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:22.865677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:22.874175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 239/502] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:22.874995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:22.921071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.938029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:22.939496 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:22.968611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:22.978289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee179987-ce71-406e-bd8c-299cbd24aaec None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.013042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.013669 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.013944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.014211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.024540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:23.029268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:23.030128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:23.031461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 240/503] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:23.035813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.051058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 264/504] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:23.052501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.141788 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8d78-1a92-466d-b689-775d7480eec8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.144125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.160276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184aa31-7f6f-4759-993e-611a12aa009c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.174928 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.175556 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.175835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.176105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.177288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:23.177591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.178963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.186719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 241/505] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:23.187708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.194960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.195594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.195879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.196149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.196890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:23.197194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.198591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.203922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.234593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 265/506] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:23.235373 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.292224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c51f-129b-4d7b-a2da-18a6b9a7daf8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.324606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.325206 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.325481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.325750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.326396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:23.326696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.328094 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.333715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850852e2-b7cf-4f03-870f-0afd116ce5c4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.335903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 242/507] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:23.336502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.373433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.374049 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.374323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.374590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.384768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.386209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.387889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:23.400876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:23.401767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:23.402590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:23.402590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.402590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:23.402590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 266/508] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:23.402865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.409110 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.439628 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321721f5-94a9-4238-a789-322d0e93f101 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.471528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.472149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.472430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.472696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.483959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.485456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:23.499632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:23.500504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:23.501313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:23.501313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.501313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:23.501313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 243/509] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:23.501528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.505449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800817-8f28-4dc6-96fa-3c5084e5a2c0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.538554 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.539171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.539475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.539757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.550321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.551804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:23.565950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:23.567426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:23.568278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:23.568278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.568278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:23.568278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 267/510] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:23.568278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.599102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22c140-06d9-4132-809f-68f85628acc8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.614202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.623867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.633859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.634463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.634746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.635017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.668492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0523c-78d4-417e-bbc4-1c09ae311a57 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.700790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.701326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.701426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.701657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.701926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.703031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.712709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.714150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:23.727982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:23.728962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:23.729804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:23.729804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:23.729804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:23.729804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 268/511] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:23.729804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.805377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 244/512] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:23.806785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.830995 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.831798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:23.835434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7c7fc-1435-4709-83d8-0acd9b654cfb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.869264 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.869847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.870122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.870397 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.871028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:23.871354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.872724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.884158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 269/513] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:23.884938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.904063 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf56e3-3897-4bcd-98ea-e4a280c8be9e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:23.936739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:23.937368 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:23.937642 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:23.937913 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:23.938542 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:23.938841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:23.940242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:23.950859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 245/514] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:23.951655 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:23.987254 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c811af-2d2a-46b8-a65f-e846ee0bf737 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.021038 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.021623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.021902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.022173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.022801 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:24.023102 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.024487 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.034827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.035201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 270/515] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:24.036586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.052025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794209fd-321e-404d-b812-7deb3df755d4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.059276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.088080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.088696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.088973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.089248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.089882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:24.090189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.091596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.103853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 246/516] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:24.104632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.141653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c89ad-f75a-4d9f-a08e-653c77885432 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.174090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.174713 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.174994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.175270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.186342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.187811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.199781 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee84ead-72c4-4c97-8d07-16b3242f3261 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:24.201519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:24.202485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:24.203791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:24.203791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:24.203791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:24.203791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 271/517] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:24.203791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.233136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.233756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.234036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.234307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.241609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.243028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:24.254525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:24.255541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:24.256398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 247/518] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:24.257293 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.266412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.275340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.308083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31cd8d-cadf-4452-b23d-9e9c762bb9de None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.341507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.342040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.342317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.342586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.343226 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.344670 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.354486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719152e5-0056-4273-adb6-3d2de0110d80 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.366763 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 272/519] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:05:24 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:24.367591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.392107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.392776 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.393055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.393331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.402467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.404274 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:24.420430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:24.421412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 19:05:24.422327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:24.428900 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 248/520] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:24.428900 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.487923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b58c-f8ff-4b8e-87f6-5ef418cacdff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.505282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.518599 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.529562 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.530521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.530864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.531206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.540395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.542459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.548235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc1a23-4976-46f6-8e0b-c2f2886a3c46 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:24.557073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:24.557977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:24.559028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 273/521] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:24.560811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.589779 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.590408 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.590709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.590980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.591786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:24.592036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.593423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.605723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 249/522] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:24.606392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.673870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af423717-04a7-40ed-b9cd-bf62336021c6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.712658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268484-f75d-45b5-a80f-03b1443f9e15 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.717529 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.718139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.718422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.718696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.719419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:24.719727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.721338 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.740571 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 274/523] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:24.740930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.741622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.756274 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.757044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.757332 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.757609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.758338 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:24.758643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.760109 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.778460 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 250/524] 10.222.0.18 () {56 vars in 1139 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:24.779583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.800578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:24.860476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b90ff2-0891-4ce5-b40b-913b25ec2304 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.883698 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18a547-3d1c-45dd-8ba5-e2aef155519b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:24.901313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.901983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.902199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.902472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.903206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:24.904061 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.905495 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.920888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:24.921520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:24.921798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:24.922147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:24.924608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 275/525] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:24.928208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.929494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:24.930923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:24.943027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:24.943889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:24.944728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:24.951153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 251/526] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:24.951153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:24.995210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.038134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3360c-dc94-44fe-97ba-49c736d85b77 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.040579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.050742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e941d3-8243-4a2e-9e8d-f2a300657aa6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.080777 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.081398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.081683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.081965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.082768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=8d7573ee32f6443892a361f012c2f204) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:25.083073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.084553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.090051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 276/527] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/role_assignments?user.id=8d7573ee32f6443892a361f012c2f204&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:25.090837 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.093719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.094346 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.094627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.094902 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.184495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.185892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.243396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.259530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cbd5-5601-49b8-ae62-14e44f78d57c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.309709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.310342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.310604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.310882 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.318637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 252/528] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:25.319366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.374993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.423955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.427262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.430737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457905b-7a09-4310-8277-17fbffcee454 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.466655 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.467184 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.467506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.467823 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.468521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:25.468826 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.470377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.494830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 253/529] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:25.494830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.499146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.565754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 277/530] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:25.565754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.621096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ae62e-dc7f-4245-9795-2184df41a6d7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.641696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.659579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.660224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.660503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.661389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.672562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.673830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.686720 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feae36-7f5f-43a5-ab6d-2dd79a11aacf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:25.693923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:25.694816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:25.696430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:25.696430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:25.696430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:25.699442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 254/531] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:25.700294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.728989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.729523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.729801 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.730072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.730789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=e7f6b6c5a3f0462d8da8b4a7afec5335) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:25.731092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.732669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.738367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 278/532] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/role_assignments?user.id=e7f6b6c5a3f0462d8da8b4a7afec5335&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:25.739195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.773185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:25.828434 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f3ef-95ec-4189-8040-2aae52251e8c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:25.869763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:25.870746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:25.871027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:25.871464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:25.884524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:25.886028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:25.905127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:25.906088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:25.906938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:25.906938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:25.906938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:25.906938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 255/533] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:25.912095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:25.990141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66663c-8615-4cc0-89f4-41151dea684d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.034820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4018c-57e5-4cc7-a74b-6be200dbb835 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.040482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:26.046956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:26.055988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.056604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.056881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.057152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.057798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:26.058101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.059568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.078828 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.079490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.079771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.080045 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.080843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:26.081144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.082213 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 279/534] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:26.087503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.093763 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:26.101813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 256/535] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:26.107102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:26.220707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf28ed-5ab8-473b-b7bc-b54585c16e85 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.265458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.266102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.266378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.267721 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.282513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.284008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.304407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aaf57-5066-42d4-9c17-562204a9d4ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:26.313830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:26.315859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:26.317935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:26.323401 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 257/536] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:26.328969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:26.344153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.344789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.345064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.345334 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.353309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.354746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:26.369433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:26.370302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:26.371149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 280/537] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:26.377034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:26.412699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:26.447983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:26.499959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20321-4482-4033-93d2-6045054f7b62 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.544587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.545114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.545389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.545659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.546349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:26.546649 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.548137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.562499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 281/538] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:26.570014 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:26.586962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71386eb2-8ebf-4927-9367-e277a93dadc9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.642204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.643132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.643578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.643973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.645065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:26.646045 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.648124 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22e2d1-7d58-4fc7-ba90-7bfdfddc33ff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.663978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 258/539] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:26.665234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f2585-79e4-462d-9025-3b3f670e90ba None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.665519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f2585-79e4-462d-9025-3b3f670e90ba None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.665793 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f2585-79e4-462d-9025-3b3f670e90ba None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.671857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 259/540] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:26.672632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f2585-79e4-462d-9025-3b3f670e90ba None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:26.733796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fabfe-f83a-418d-9078-c7e9db0c8ca5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.752748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:26.768351 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:26.782356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.783015 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.783320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.783585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.784354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=fc1c6323c0a34d8f9d36fdc95712d26f) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:26.784663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.785319 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f2585-79e4-462d-9025-3b3f670e90ba None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:26.786116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a8029-7bcc-41a6-8b64-996e3b30a461 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.793793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 282/541] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:26 2022] GET /identity/v3/role_assignments?user.id=fc1c6323c0a34d8f9d36fdc95712d26f&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:26.795632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.795862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.796138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.822252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:26.823475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:26.824252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:26.824980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:26.825357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:26.825791 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:26.826742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=0b094a1d708f4b02b83405e6befbed9f) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:26.827157 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:26.828777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:26.836478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 260/542] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:26 2022] GET /identity/v3/role_assignments?user.id=0b094a1d708f4b02b83405e6befbed9f&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:26.909020 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 283/543] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:27.205638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:27.313526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71adbd-ed2e-49ed-becb-2ce2e043cef0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:27.361805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:27.362708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:27.362994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:27.363346 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:27.375765 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:27.381834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:27.384072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:27.385969 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 261/544] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:27.391311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:27.489917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69baf01-5f0d-49bc-bd13-7c9080f95915 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:27.501956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:27.542840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:27.543476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:27.543753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:27.544027 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:27.544696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:27.544998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:27.546413 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381bfbc-7474-43b6-ad29-f76aeb41c6a3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:27.558921 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 284/545] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:27.564853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:27.589859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:27.613593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dc019-2d2e-4363-8bfd-5dbec263734e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:27.614700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dc019-2d2e-4363-8bfd-5dbec263734e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:27.615187 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dc019-2d2e-4363-8bfd-5dbec263734e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:27.616918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 285/546] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:05:27.622906 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:27.623195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:27.623500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:27.652746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:27.654467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:27.731135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198ac5f-af08-4697-ab2e-4163d96106b7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:27.744479 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 286/547] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:27.793191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:27.793834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:27.794118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:27.794395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:27.808353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:27.809825 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:27.833657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:27.834615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:27.835470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:27.835470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:27.835470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:27.835470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 262/548] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:27.840439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:27.957892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:27.967602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c88bc-4336-45d4-9670-bceef166b951 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.014418 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.015028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.015324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.015653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.016855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:28.017376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:28.019604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:28.031447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 287/549] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:28.037143 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.118797 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.140180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:28.191639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfa8e9-8718-4bae-8dd3-336a7745ed3a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.230939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.232265 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.232689 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.233137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.241852 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:28.244379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:28.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:28.259108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:28.260343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:28.263116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 263/550] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:28.269546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.386096 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63143360-9d40-40b4-898f-0f15a667a4ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.390172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0aae-5e53-49b4-8931-fd64cdd219e9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.423444 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:28.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c198ccd-5275-42b7-b348-90df814cc835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.436160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198ccd-5275-42b7-b348-90df814cc835 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.436477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198ccd-5275-42b7-b348-90df814cc835 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.436757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198ccd-5275-42b7-b348-90df814cc835 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.438790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.439681 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.440170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.440630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.441565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:28.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:28.444478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:28.449205 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c198ccd-5275-42b7-b348-90df814cc835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6c198ccd-5275-42b7-b348-90df814cc835 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:28.454946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:28.456086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:28.456907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:28.459160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 288/551] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:28.464760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a854b-b547-4b8c-91a4-8f63b997cb52 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.465295 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a854b-b547-4b8c-91a4-8f63b997cb52 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.465793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a854b-b547-4b8c-91a4-8f63b997cb52 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.467579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 289/553] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:28.467686 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 264/553] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:28.469457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.470352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a854b-b547-4b8c-91a4-8f63b997cb52 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.494656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:28.593123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a854b-b547-4b8c-91a4-8f63b997cb52 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.597981 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970b797-7155-44da-9f0e-13088e625b1a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.629859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.630468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.631043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.631153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.636267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.637116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.637613 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.638101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.638964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:28.639516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:28.641263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e4878-689f-4b08-8659-b961890fd9c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:28.654458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 265/554] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:28.656136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.656717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.657295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.676559 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:28.678146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:28.706413 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:28.708225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:28.723901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:28.728597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:28.757926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 266/555] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:28.759107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.826962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 290/556] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:28.834546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:28.877354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5f187-c3df-4c22-9a2b-841ff5953bba None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:28.945000 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:28.946064 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:28.947744 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:28.948245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:28.963841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:28.965302 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:28.989907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:28.990884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:28.991777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:28.991777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:28.991777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:28.991777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 267/557] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:28.998526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.006551 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f432fa-4490-465f-9fc7-56dde307ef28 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.027461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.048661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.049780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.050263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.050737 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.062503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.064302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.067227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:29.082370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:29.084322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:29.085195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:29.085195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:29.085195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:29.087265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 291/558] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:29.096408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.173212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b7b1-b53f-4c2a-8bf9-a4dce415586a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.211030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.211677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.211958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.212232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.212880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:29.213182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.214574 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.227200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc40e2a-b5c9-489d-8beb-c0a96ba4cefa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.227535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 268/559] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:29.228377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.267492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.268094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.268376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.268647 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.269678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:29.270525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.272340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.286466 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 292/560] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:29.287311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.316728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.320349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.335318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed5c9f-4b15-4a0a-b4e4-1297cf0b351f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.372033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.372939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.373366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.373798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.385161 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:29.390335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:29.391450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:29.392374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:29.394448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 269/561] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:29.395513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.402206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652047-2dc9-4b56-b2b6-5bdbc3e9a821 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.443412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.444092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.444323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.444594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.453994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.455610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:29.470222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:29.471421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:29.472389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 293/562] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:29.473024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.515667 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07bab3-a866-451d-8801-0d5e774b918a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.553756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.554308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.554593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.554872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.561630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.563063 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:29.576079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:29.576973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:29.577841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 270/563] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:29.578886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.590417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.592613 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.602135 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a6199-4848-40e6-9792-29727e1a4204 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.642235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.642839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.643119 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.643423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.644091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:29.644397 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.646326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ff9ca-2076-4fbe-a58c-60a0166806de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.653958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 294/564] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:29.655092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64e017-5329-421d-b472-ca3fe7a35c4d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.655423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64e017-5329-421d-b472-ca3fe7a35c4d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.655703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64e017-5329-421d-b472-ca3fe7a35c4d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.656892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 295/565] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:29.657587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64e017-5329-421d-b472-ca3fe7a35c4d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.690375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd92bf-e410-40c2-8b6f-f6d9cb442732 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.725482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.726093 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.726358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.726630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.727368 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:29.727670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.729099 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac258c-123f-4a4d-a4b7-39b4f2ffdb62 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.745586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 271/566] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:29.746956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1c7d5-7c2e-4e66-b34c-10e30b04505c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.747247 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1c7d5-7c2e-4e66-b34c-10e30b04505c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.747547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1c7d5-7c2e-4e66-b34c-10e30b04505c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.748847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 272/567] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:29.749463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1c7d5-7c2e-4e66-b34c-10e30b04505c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.761883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64e017-5329-421d-b472-ca3fe7a35c4d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.798249 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.798854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.799174 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.799445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.800134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:29.800439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.802423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f4d3b-412c-4c74-b51a-f4346d4440ca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.812006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.814126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 296/568] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:29.815108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.815420 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.815695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.833159 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:29.834359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:29.841405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:29.854094 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1c7d5-7c2e-4e66-b34c-10e30b04505c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:29.887587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:29.888202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.888479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.888745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.898594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:29.899177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 297/569] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:29.899935 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:29.900078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8b5ec802-b82f-4bba-89a4-82b319340061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:29.914923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:29.920650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 273/570] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:29.921763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:29.943026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:29.944263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:30.011231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defee66-3b0c-4453-954f-5510c04513c1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.019486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 274/571] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:30.019486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.072540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.075631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.077847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.078133 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.078431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.095454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.097510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.100821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:30.120238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:30.121262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:30.122182 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:30.122182 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:30.122182 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:30.125103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 298/572] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:30.126110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.141362 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e750966-ca51-4c6f-a5a0-6ee088e769dc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.179233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.180206 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.180594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.180878 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.181605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=cb269fa787664ebead0e2e9fd21ce219) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:30.181915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.183562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.189023 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 275/573] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/role_assignments?user.id=cb269fa787664ebead0e2e9fd21ce219&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:30.191636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.245101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaac9d9-d132-434f-965c-cdf62855b94f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.293267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.293887 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.294165 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.294435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.295088 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:30.295494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.296970 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.310937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 299/574] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:30.311889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.324804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef7a8-1fdd-49e2-a33c-23ff80afb8a9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.336451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.369234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.370952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.371235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.371586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.372514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:30.372871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.374314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.374431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.389639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 276/575] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:30.391316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.424855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c71d1-9bb7-46ac-8e43-85736e0ed3b5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.460489 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.461157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.461379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.461655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.471991 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:30.476955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:30.477873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:30.478705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 300/576] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:30.479432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.499638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde5ffe-fbd9-4247-b611-b467da01161f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.535906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.536584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.536861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.537128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.547734 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:30.553071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:30.554153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:30.554992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 277/577] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:30.555705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.590190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.597374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65328210-3810-4165-994e-a2fe80b8e1bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.618473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.638518 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.639142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.639448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.639720 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.647222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.648755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:30.660107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:30.661361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:30.662300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 301/578] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:30.662886 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.674282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f3060-d668-4bac-a0b2-e5a851c2b8a0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.715714 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.716346 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.716624 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.716896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.724478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.725946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:30.737148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:30.738470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 278/579] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:30.740294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.773215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74895900-6dd4-433a-b0c8-4b741be48d9a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.814971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.815607 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.815884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.816153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.816793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:30.817661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.819024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.832488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 302/580] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:30.832488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.840127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.850041 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9753-a81f-4ba0-83e7-3931ef0be938 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.870814 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:30.885760 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.886370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.886648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.886919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.887604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:30.887906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.889290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:30.899333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 279/581] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:30.899333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:30.945997 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53ade-dadb-494e-a684-529418867ab7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:30.984035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:30.984646 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:30.984926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:30.985197 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:30.985925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:30.986224 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:30.987929 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.002132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68141a1a-65d7-4e42-95e3-75261292464b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.006083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 303/582] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:31.006893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.038204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.038808 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.039084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.039382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.040065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:31.040372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.041881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.055319 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 280/583] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:31.056284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.075277 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.128002 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.137814 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c32e3-26a4-49c8-88fa-d864f1a539e1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.161999 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd261048-baaa-48ae-ad37-9111d67905ae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.174616 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.175257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.175550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.175816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.186111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.187581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.196012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.196632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.196910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.197195 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:31.202277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:31.203208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:31.209242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:31.209242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:31.209242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:31.209242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 304/584] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:31.209242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.213573 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.215089 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:31.230991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:31.231997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:31.232883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:31.232883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:31.232883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:31.232883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 281/585] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:31.232883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.296354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.316679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f40db-d1d5-4ddf-a308-c47438b03dbb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.338892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18631c8-b0fe-41ee-84fa-9cc7dc8c4b6f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.351067 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.351942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.352407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.352676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.364483 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.365900 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.373187 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.373426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.373837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.374114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.374383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/8d7573ee32f6443892a361f012c2f204/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:31.383927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:31.385051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:31.386132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:31.386132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:31.386132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:31.389127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=8d7573ee32f6443892a361f012c2f204, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.390556 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.393252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 305/586] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:31.394767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.407359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 282/587] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:31 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/8d7573ee32f6443892a361f012c2f204/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:31.408232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.528562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b95319-8826-49cf-bf72-3f13522acfef None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.548787 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.554108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae2987-9e74-4c67-a878-68b77fa3780f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.575949 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.576921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.577051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.577429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.578212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:31.578555 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.580030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.595098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 283/588] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:31.596191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.603066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.603941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.604214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.604483 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.605157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:31.605474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.606818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.621544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 306/589] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:31.622291 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.699403 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.715165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f6c3d-0cbf-438e-8251-ec8bff8d01d8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.752669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f98f9-6f74-4dfe-84bd-d6d2e5c81dc2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.763061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.764337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.765097 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.765576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.766582 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:31.767099 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.768792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.781349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 284/590] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:31.782471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.789712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.790430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.790780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.791122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.799695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.802104 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:31.815426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:31.816497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:31.817490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 307/591] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:31.818075 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.823938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.892981 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa3613-8080-40a7-94e6-5ce9a99c7146 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.929571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.930487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.930951 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.931432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.938968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.939917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0a5f4-ee22-4333-8b1b-14e4bee15894 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:31.940864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:31.952924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:31.953863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 19:05:31.954789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:31.962359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:31.962521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 285/592] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:31.963279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:31.978971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:31.979890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:31.980273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:31.980644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:31.988328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:31.990753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:32.003652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:32.005219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:32.006827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 308/593] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:32.009984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.073509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.080361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320693a-5147-4f48-844a-21ba51d6b3ce None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.115854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.116467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.116748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.117033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.117748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:32.118055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.119515 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.130411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73617b3-7d79-48b8-a157-007e464ad8ff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.138997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 286/594] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Sep 16 19:05:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:32.139939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.168869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.169788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.170347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.170852 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.172000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:32.172499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.174280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.195127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 309/595] 10.222.0.18 () {56 vars in 1139 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:32.196321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.243552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.259275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d62e8-f830-495e-9543-d17b16f24f01 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.294184 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.294801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.295080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.295382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.296085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:32.296389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.297866 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.311911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 287/596] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:32.312794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.324241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98aef1-e700-40c0-bc34-a4fb6c947cea None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.329374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.368086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.369002 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.369462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.369889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.383226 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.385102 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:32.403108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:32.404354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:32.405257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:32.405257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:32.405257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:32.407149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 310/597] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:32.408138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.422633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac6de-9598-4479-ab2d-06b137896a57 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.456807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.457479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.457991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.458272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.468922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.470365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:32.485135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:32.486077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:32.486994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:32.486994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:32.486994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:32.486994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 288/598] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:32.488147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.513721 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.527510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72132e68-f5e5-48b8-91e0-bc6ab6c5cf63 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.562117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e756637-daaa-4d4c-a8f3-6d977aa8828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.562774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e756637-daaa-4d4c-a8f3-6d977aa8828b None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.563192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e756637-daaa-4d4c-a8f3-6d977aa8828b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.563547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e756637-daaa-4d4c-a8f3-6d977aa8828b None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/e7f6b6c5a3f0462d8da8b4a7afec5335/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.567936 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.578832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e756637-daaa-4d4c-a8f3-6d977aa8828b None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=e7f6b6c5a3f0462d8da8b4a7afec5335, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.580306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e756637-daaa-4d4c-a8f3-6d977aa8828b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.594247 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 311/599] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:32 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/e7f6b6c5a3f0462d8da8b4a7afec5335/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:32.595254 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d1df0-b7c3-4de5-8219-7fcb803bdebf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.595568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d1df0-b7c3-4de5-8219-7fcb803bdebf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.595844 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d1df0-b7c3-4de5-8219-7fcb803bdebf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.597255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 312/600] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:32.597798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d1df0-b7c3-4de5-8219-7fcb803bdebf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.634457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb57e-b636-4601-bb8e-d59ff046f9a6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.677248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.677988 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.678287 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.678607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.679279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:32.679611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.681219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f7664-23fb-4bf6-9b0f-ef7e75de339c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.695592 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 289/601] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:32.696906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.697257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.697531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.720255 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:32.721458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:32.750702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d1df0-b7c3-4de5-8219-7fcb803bdebf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.792174 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.793123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.793616 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.794101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.794959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:32.795520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.797349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.805449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.815186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 313/602] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:32.816074 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.817980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 290/603] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:32.821073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.882543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:32.927247 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678467e0-17a3-4554-8b9f-fcfdfc7c04cb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.955344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ab799-f928-475b-9114-73f8fa0677b6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:32.963238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.964031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.964406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:32.964772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:32.971989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:32.973927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:32.985887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:32.987012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:32.988457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:32.990606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 291/604] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:32.998312 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:32.999124 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:32.999435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:32.999767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.000273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.009028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.010672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:33.023689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:33.024750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:33.025771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:33.028348 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 314/605] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:33.033449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:33.139656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:33.187143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187824-2ea2-4b0d-9cc9-34d48e563f36 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.200853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44fafd-d286-45d2-9a16-05ecf1c0257f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.220226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:33.224915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.225525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.225802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.226071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.226758 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:33.227061 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.228509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.242517 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 315/606] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:33.243767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:33.248656 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.249570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.250044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.250500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.251387 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:33.251882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.254016 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.271657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 292/607] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:33.272938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:33.364546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a82ffe-07a7-4d71-826a-706b29ace718 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.391453 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:33.403450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fba77-f445-4b2b-88f3-5f600ed1d18f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.416912 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.418245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.418706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.419436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.435263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:33.441698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:33.442834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:33.443965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:33.446601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 316/608] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:33.447556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:33.454084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.454835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.455187 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.455563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/0b094a1d708f4b02b83405e6befbed9f/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.471916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=0b094a1d708f4b02b83405e6befbed9f, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.474214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.490603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 293/609] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:33 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/0b094a1d708f4b02b83405e6befbed9f/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:33.491728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:33.520673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:33.573158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062eda8-fd3d-4289-9e1e-de4e70f3e294 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.614244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf28e3-12c5-4512-be05-b8ff7620620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.615322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf28e3-12c5-4512-be05-b8ff7620620c None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.615807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf28e3-12c5-4512-be05-b8ff7620620c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.616264 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf28e3-12c5-4512-be05-b8ff7620620c None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/fc1c6323c0a34d8f9d36fdc95712d26f/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.635135 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf28e3-12c5-4512-be05-b8ff7620620c None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=fc1c6323c0a34d8f9d36fdc95712d26f, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.637168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf28e3-12c5-4512-be05-b8ff7620620c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.662905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 317/610] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:33 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/fc1c6323c0a34d8f9d36fdc95712d26f/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:33.663316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df751c9-6c77-49b7-9c61-ee2e1a97f252 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.732187 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715304-d886-4316-895c-4efba805701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.733204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715304-d886-4316-895c-4efba805701e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.733697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715304-d886-4316-895c-4efba805701e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.734167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715304-d886-4316-895c-4efba805701e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.735054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715304-d886-4316-895c-4efba805701e None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:33.735574 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715304-d886-4316-895c-4efba805701e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.737595 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715304-d886-4316-895c-4efba805701e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.755249 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 294/611] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:33.760117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e4ac7-b14f-4fc5-802b-e69776d215ef None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.760488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e4ac7-b14f-4fc5-802b-e69776d215ef None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.760837 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e4ac7-b14f-4fc5-802b-e69776d215ef None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.761150 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715304-d886-4316-895c-4efba805701e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:33.764094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 318/612] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:33.769008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40386179-0b47-4034-857f-5963c26fea11 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.769303 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40386179-0b47-4034-857f-5963c26fea11 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.769684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40386179-0b47-4034-857f-5963c26fea11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.791040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-40386179-0b47-4034-857f-5963c26fea11 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:33.792443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-40386179-0b47-4034-857f-5963c26fea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:33.830804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:33.903463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 319/613] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:33.916500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:33.923908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715304-d886-4316-895c-4efba805701e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:33.964361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:33.965017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:33.965295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:33.965576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:33.977830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:33.979334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:33.997074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:33.998853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:34.000268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:34.000268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:34.000268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:34.000268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 295/614] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:34.003784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40386179-0b47-4034-857f-5963c26fea11 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.117037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40386179-0b47-4034-857f-5963c26fea11 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:34.154300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:34.154823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:34.155107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:34.155409 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:34.156051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:34.156415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:34.157783 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:34.172869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 320/615] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:34.180471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.234977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.257836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:34.376743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db0dc3-a21d-4d71-8b30-50075912bd5e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:34.423486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d64be-dfac-4395-9c00-db92157275b1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:34.423728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:34.424371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:34.424651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:34.424925 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:34.441341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:34.442872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:34.451625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:34.468018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:34.469428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:34.470388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 296/616] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:34.477054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.494440 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:34.495087 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:34.495389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:34.495667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:34.512921 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:34.525750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:34.527145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:34.529380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 321/617] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:34.533152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.655956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:34.661807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a1810-fd58-419f-954c-60e90d24e759 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:34.710119 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20d0f-9b71-4fcc-9ea9-bf854f8ea7cc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:34.744606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:34.745227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:34.745507 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:34.745751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:34.746486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:34.746790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:34.748269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:34.784775 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:34.785380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:34.786216 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:34.786491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:34.787162 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:34.787488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:34.788847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:34.790578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 297/618] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:34.791488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.799750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 322/619] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:34.806557 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:34.898503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:34.989502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9492-b766-4e64-b233-633419280392 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:34.998989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d2c33-6dce-40ef-bc5d-5409d51b6795 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.026120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.026895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.027274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.027566 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.038368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:35.040158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.042362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:35.059086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:35.060090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:35.060985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:35.060985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:35.060985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:35.066177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 323/620] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:35.075364 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.075959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.076239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.076672 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.076995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.091433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.093156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:35.110405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:35.111562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:35.112558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:35.112558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:35.112558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:35.114963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 298/621] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:35.122706 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.195246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c7413-0628-40ce-9ad5-0be403105f2b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.233674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.234570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.235391 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.235809 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.236781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:35.237359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.239325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.252767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 324/622] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:35.257839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.349691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f5609-152e-4ac2-a82a-c93b1715126d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.373881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:35.374412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:35.433264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.433931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.434271 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.434540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.435189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:35.435510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.439634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.459264 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bb8a8-8394-474d-beac-a7799837ea2d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.473950 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 299/623] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:35.480341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.502898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.503854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.504132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.505189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.513602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.515569 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-68ec63ee-f68e-411a-90a4-1ee5e609e47b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:35.530431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:35.531437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:35.532393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 325/624] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:35.533506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b209a1-7d4e-4100-99e8-5f962af1ae6d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.533786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b209a1-7d4e-4100-99e8-5f962af1ae6d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.534412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b209a1-7d4e-4100-99e8-5f962af1ae6d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.536331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 326/625] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:35.537183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b209a1-7d4e-4100-99e8-5f962af1ae6d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.621707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f4965-ea1e-4100-8637-f07f17c9b5dd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.657660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b209a1-7d4e-4100-99e8-5f962af1ae6d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.668436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.669083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.669363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.670107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.674300 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:35.680317 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.681785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-67e13626-8626-4f3c-ac81-005b9a361c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:35.696476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:35.698519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:35.700657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:35.701650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.701822 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 300/626] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:35.701933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.701933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.701933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.702236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.702236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.702550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:35.702685 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.702810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.702810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.722123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 327/627] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:35.723108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:35.723317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.724510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:35.758194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:35.825275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 301/628] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:35.825275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:35.874517 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a53ba3-8ca4-4a6f-b791-cbd3992fa671 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.934385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:35.935265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:35.935884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:35.936314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:35.938253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7b9a6-f250-42dc-a54d-2c61cc673dcf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:35.940398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:35.941076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:35.943854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:35.968572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 328/629] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:35.975321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.022209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.038136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013d89-b3e9-4464-9415-1c1804c7acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.038755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013d89-b3e9-4464-9415-1c1804c7acef None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.039036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013d89-b3e9-4464-9415-1c1804c7acef None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.039320 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013d89-b3e9-4464-9415-1c1804c7acef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.087666 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.134194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46013d89-b3e9-4464-9415-1c1804c7acef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:36.135749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46013d89-b3e9-4464-9415-1c1804c7acef None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:36.136736 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd6da-f2f3-4404-96ed-d68b686412e3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.179658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6d42d-5aed-418c-8ea0-dd9750e3955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.180345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6d42d-5aed-418c-8ea0-dd9750e3955e None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.180629 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6d42d-5aed-418c-8ea0-dd9750e3955e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.180905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6d42d-5aed-418c-8ea0-dd9750e3955e None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/cb269fa787664ebead0e2e9fd21ce219/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.198985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df6d42d-5aed-418c-8ea0-dd9750e3955e None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=cb269fa787664ebead0e2e9fd21ce219, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:36.200502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df6d42d-5aed-418c-8ea0-dd9750e3955e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:36.217819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 329/630] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:35 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/cb269fa787664ebead0e2e9fd21ce219/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:36.219130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6b332-99a2-457a-a22f-688db7313645 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.223537 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6b332-99a2-457a-a22f-688db7313645 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.223860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6b332-99a2-457a-a22f-688db7313645 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.225297 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 330/631] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:36.226079 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6b332-99a2-457a-a22f-688db7313645 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.275550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 302/632] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:36.277998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.278284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.278561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.298322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:36.299647 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:36.335181 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.357260 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6b332-99a2-457a-a22f-688db7313645 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.363591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.380998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 303/633] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:36.382309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.414683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bdfb0-1891-4e69-886e-b108be81c7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.415252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bdfb0-1891-4e69-886e-b108be81c7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.415570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bdfb0-1891-4e69-886e-b108be81c7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.415841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bdfb0-1891-4e69-886e-b108be81c7e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.430547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bdfb0-1891-4e69-886e-b108be81c7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e33bdfb0-1891-4e69-886e-b108be81c7e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:36.435506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:36.436435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:36.437302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 331/634] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:36.438500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28538b6a-433a-4769-89ba-7f1e44321a23 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.438500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28538b6a-433a-4769-89ba-7f1e44321a23 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.438500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28538b6a-433a-4769-89ba-7f1e44321a23 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.439711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 332/635] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:36.444336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28538b6a-433a-4769-89ba-7f1e44321a23 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.524877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acfd76-9d23-4faf-a36d-8ee1e595f2eb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.561992 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28538b6a-433a-4769-89ba-7f1e44321a23 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.574058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.575309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.575803 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.576529 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.589836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:36.591427 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:36.602061 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.602966 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.603500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.603976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.604841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:36.605433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:36.607054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8a667ca7-23b9-4e55-a3ee-9f9c5a0efb1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:36.607162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:36.608053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:36.608932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:36.608932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:36.608932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:36.608932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 304/636] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:36.609914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.610192 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.610469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.625345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 333/637] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:36.626660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.630417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.633153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:36.634385 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:36.679281 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.723313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 305/638] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:36.723313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.744881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ae77-8e40-4917-beae-0991031edf1d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.782728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.783359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.783642 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.783916 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.784554 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:36.784911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:36.786568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:36.800492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 334/639] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:36.802253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.832590 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1abe45-dcfd-4689-8c4c-81f6de31bb2b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.869580 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.870263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.870541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.870805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.881674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.881812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:36.883135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:36.899225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:36.900228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:36.901212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:36.901212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:36.901212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:36.901212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 306/640] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:36.901954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:36.925161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a9dc4-ad37-44c7-83fd-7603c2c3bacb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:36.931936 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:36.967392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:36.968029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:36.968309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:36.968584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:36.981890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:36.988394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:36.990527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 335/641] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:36.992589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.024816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1033050-094d-4476-8fc3-848cc33c59a4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.063322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41717be8-279d-4830-8961-e484747b5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.064081 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41717be8-279d-4830-8961-e484747b5114 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.064459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41717be8-279d-4830-8961-e484747b5114 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.064825 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41717be8-279d-4830-8961-e484747b5114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.072170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41717be8-279d-4830-8961-e484747b5114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.073716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41717be8-279d-4830-8961-e484747b5114 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-41717be8-279d-4830-8961-e484747b5114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:37.085673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:37.086660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:37.087589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 307/642] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:37.088584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41717be8-279d-4830-8961-e484747b5114 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.116447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7df035-6f40-4d1e-a8c8-e874dea2f33f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.140035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.153341 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.153940 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.154216 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.154485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.155114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:37.155441 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.156804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.169059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 336/643] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:37.169903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.182023 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.204288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41717be8-279d-4830-8961-e484747b5114 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.244664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.245615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.246169 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.246454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.247139 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:37.247476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.249022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.264288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 308/644] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:37.264288 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.287173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434e44-688b-44eb-bc03-c5b9aa0a06cf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.334573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.335544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.335722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.335942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.347498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:37.352984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:37.353896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:37.354860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:37.357521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 337/645] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:37.358476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.383603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf27e61-6dbd-4463-acd0-55fca43950bf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.415695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.423343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.423501 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.423856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.424194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.425016 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:37.425353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.426900 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.444603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 309/646] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:37.445479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.449049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.484270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42423-fca2-4edf-be0b-b6339706f879 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.525184 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.525819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.526103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.526379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.534574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.536845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:37.549942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:37.550930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 338/647] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:37.552276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.558221 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9e1f8-7962-4e6d-8134-90136e035f05 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.594493 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.595132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.595511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.595848 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.606960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.608806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:37.624756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:37.625722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:37.626597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:37.626597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:37.626597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:37.631516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 310/648] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:37.632470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.675861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252442d8-49c1-48d0-99d4-57ea076d7c36 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.678507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.721091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.725143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.725822 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.726138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.726542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.727250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:37.727651 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.729103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.739937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 339/649] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:37.741077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.770676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9196-331e-4b11-b2ad-4780c3a5009e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.806353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.807506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.807968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.808570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.810054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=8d7573ee32f6443892a361f012c2f204) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:37.810544 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.813012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.821038 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 311/650] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/role_assignments?user.id=8d7573ee32f6443892a361f012c2f204&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:37.822002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.853366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa59c-71e3-479d-977e-287667564dda None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:37.892146 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:37.892865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:37.893217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:37.893561 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:37.894443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:37.894866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:37.896956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:37.917298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 340/651] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:37.917986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:37.945549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:37.972620 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f902f6-9c2a-4b60-85a0-7d5f0ce18a2e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.001627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.023586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.024499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.024877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.025269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.026237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:38.026528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.028097 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.043916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 312/652] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:38.045591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.047773 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72242-15bd-494c-a851-194c9c2ce2ce None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.096585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.097332 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.097687 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.098061 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.109052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.110429 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:38.126952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:38.128012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:38.128990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:38.128990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:38.128990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:38.128990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 341/653] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:38.128990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.178216 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9508ef-45ae-4970-be3e-415510e566ae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.216719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.217331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.217607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.217877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.231092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.260176 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0181709-95d5-4a56-b574-3a19a16ca3aa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.278300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.300399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.301649 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.302055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.302330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.303194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.305095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.310331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.311810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:38.324585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:38.325558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:05:38.326505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:38.330314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 342/654] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:38.331143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.430396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 313/655] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:38.431560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.453710 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daba439-ba1a-42e1-9de2-7434a6bf1d10 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.493789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.494387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.494662 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.494938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.495663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:38.495959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.497324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.498781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.515491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 343/656] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:38.515491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.536793 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.556406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18c8e6-c25a-4d70-9a8f-052a6ea94671 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.593665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.594438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.594725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.594996 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.595759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:38.596066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.597484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7107a1-d530-4954-b809-3e24b2f5de50 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.616540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 314/657] 10.222.0.18 () {56 vars in 1137 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:38.617750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9969348-8609-4d21-b4f3-4c905bb46b40 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.618035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9969348-8609-4d21-b4f3-4c905bb46b40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.618311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9969348-8609-4d21-b4f3-4c905bb46b40 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.623566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 315/658] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:38.624342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9969348-8609-4d21-b4f3-4c905bb46b40 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.632242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9b8d4-3f9f-4424-b463-34b9ce3867f2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.668874 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.669501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.669781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.670054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.680963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.682374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:38.697637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:38.698650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:38.700056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:38.700056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:38.700056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:38.700056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 344/659] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:38.700056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.734678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9969348-8609-4d21-b4f3-4c905bb46b40 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.751655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.771196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.771862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.772145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.772417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.779759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.781210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.782989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6de177d0-3884-4710-9256-becd6bac4afc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:38.794540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:38.795545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:38.796469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 316/660] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:38.797082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.797082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.797763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.815692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:38.815960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ae308-3cf6-4227-a86d-cb6b6c2464a0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:38.816951 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:38.852186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:38.852734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:38.853016 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:38.854066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:38.895441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 317/661] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:38.895441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:38.939705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:38.941810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:38.991352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.011541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b6a8-b8ba-42ab-9667-3773b41cbe52 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.048183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.054541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 345/662] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:39.054541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.063390 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.064207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.064637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.065061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.065846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:39.066317 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.069431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.090648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 318/663] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:39.092353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.170813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe8c7-1f07-4b22-a782-dc373e7e7d66 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.207450 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.208093 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.208455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.208827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.209546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:39.209848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.211281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.224573 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2ccfe-8153-479d-ba51-aea905dd832b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.233419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 346/664] 10.222.0.18 () {56 vars in 1139 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:39.233419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.237619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.267716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.268496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.268904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.269286 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.280814 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.282267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:39.296363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:39.297454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:39.298337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:39.298337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:39.298337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:39.298337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 319/665] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:39.298337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.343642 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.353176 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0469d9-6310-4cc6-8c8a-aeee5815db51 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.389035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.389648 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.389931 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.390205 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.397676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.399068 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.406646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf501f5-9632-4b6c-a5e9-4b171c979faf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:39.410192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:39.411325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:39.412445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:39.413755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 347/666] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:39.414615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.445426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.446049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.446326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.446596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.456581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:39.462503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:39.463671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 320/667] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:39.464699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.486903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.526711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6f4d2-ff6e-42a9-821d-d9d26b3f9242 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.564541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.565209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.565483 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.565753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.570925 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.574105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68760d6-f4f0-48a5-9269-bff51620daac None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.609424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.610067 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.610294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.610566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.611201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:39.611528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.613146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.630112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 321/668] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:39.630112 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.647225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.648698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.715542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.734443 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b18af-0de5-498f-9d0d-92904f8415f0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.763685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 348/669] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:39.764855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.767490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.768096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.768396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.768644 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.769331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:39.769632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.771595 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.782803 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 322/670] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:39.783631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.805837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:39.878969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd995b-5128-442f-82d9-fd9902784eaf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.890248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76a69c-17e4-4cd3-87d8-27eea0b1c77b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:39.914392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.914992 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.915264 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.915560 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.916267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:39.916570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.918198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.930698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 349/671] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:39.930698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.931880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:39.932587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:39.932937 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:39.933279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:39.940275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:39.941865 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:39.953042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:39.954085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:39.955260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:39.956212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 323/672] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:39.956701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:39.966555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.038692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.053744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3a49a-0593-4f70-9e67-f74af7d12a2e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.059197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdad6f9-8e02-428a-bf1c-cebb06f58c5e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.094311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.094965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.095245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.095549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.105204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.106222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.107339 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.107687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.108815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=e7f6b6c5a3f0462d8da8b4a7afec5335) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:40.109077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.109270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.110716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.110846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.127546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 324/673] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/role_assignments?user.id=e7f6b6c5a3f0462d8da8b4a7afec5335&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:40.127546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:40.127824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:40.128898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:40.130536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:40.130536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:40.130536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:40.132152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 350/674] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:40.132786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.230195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.254965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aead12-b025-45fc-a37f-7b2038f70d1e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.266983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbef93-2bea-4e11-914c-46fbd2015bb5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.298112 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.298740 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.299021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.299316 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.300053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:40.300360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.301867 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.306793 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.307546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.307891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.308238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.320296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 351/675] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:40.320296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.328355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.329792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.333054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a2dbafc4-909c-41f1-a0ed-d432377e32b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:40.345500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:40.346629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:40.347646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:40.347646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:40.347646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:40.347860 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 325/676] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:40.349175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e468-326f-4968-98b5-4c231cf9c63e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.349469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e468-326f-4968-98b5-4c231cf9c63e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.349748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e468-326f-4968-98b5-4c231cf9c63e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.351104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 326/677] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:40.355838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e468-326f-4968-98b5-4c231cf9c63e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.435225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f7c15-ab9b-41ab-a209-33fe35560858 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.470319 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e468-326f-4968-98b5-4c231cf9c63e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.472708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.473455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.473733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.474003 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.474639 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:40.474938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.476330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.482944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.489501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 352/678] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:40.490438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.516924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.517615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.517963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.518305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.519233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=0b094a1d708f4b02b83405e6befbed9f) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:40.519699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.521697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a5a6a-4576-4558-9632-51caed0a6df6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.531750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 327/679] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/role_assignments?user.id=0b094a1d708f4b02b83405e6befbed9f&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:40.534118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.534630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.535697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.561658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:40.563009 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:40.608138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9d3df-25a8-48ad-b1a0-8523e9444399 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.641757 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.659829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.660041 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 328/680] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:40.661456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.661568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.662043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.662512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.663508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:40.664009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.665965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.682329 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 353/681] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:40.687845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.756424 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:40.800555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52f39b-bd0f-4c6a-8482-8b58f51f979e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.810973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9c129-436b-4715-9735-9c25adf01ed5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:40.843453 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.844552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.845187 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.845575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.853412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:40.853554 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.854044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:40.854322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:40.854593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:40.854961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.863835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:40.865431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:40.868507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:40.869430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:40.870377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 354/682] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:40.877075 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:40.883861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:40.884755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:40.885648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:40.886190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 329/683] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:40.892912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:40.962294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.012768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c4c6b-33a1-4339-876c-d0ebb5eb9de3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.031679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.033410 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc1e89-464c-4f7c-9699-6aac72075b96 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.056320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.057415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.057866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.058516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.062731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:41.063209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.065024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.081681 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.082924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.083335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.083717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.084533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:41.084944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.086475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.089999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 355/684] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:41.091577 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.104380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 330/685] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:41.105719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.213256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd1e67-3dde-41a3-a66e-bf2982ddf748 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.224600 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.231959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d182f64-8dc0-4c03-8e80-996432273471 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.254847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.255768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.256679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.257098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.272878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.273632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.274128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.274422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.285265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:41.291090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:41.292217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:41.293251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:41.299641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 356/686] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:41.300626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.345125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.350743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.353369 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.418368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44d7b6-ec12-46dd-8695-4271b8884f80 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.458063 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.459391 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.459815 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.460386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.461396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=fc1c6323c0a34d8f9d36fdc95712d26f) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:41.461849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.464517 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.473942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 357/687] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/role_assignments?user.id=fc1c6323c0a34d8f9d36fdc95712d26f&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:41.475661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.485006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 331/688] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:41.491819 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.506719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.608781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.613805 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c011314-e655-49c4-8496-97b0fe53df77 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.664153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.664839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.665120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.665390 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.666064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:41.666366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.667761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.679411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 358/689] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:41.685559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.696160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bffbea-1f7c-40fb-9576-3bfe5ef9a745 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.747211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.747906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.748188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.748461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.762547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.764413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:41.781912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:41.782869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:41.783813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:41.783813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:41.783813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:41.785493 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 332/690] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:41.791719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.812905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2d5f2-cc02-4de9-bd9d-4adb34706e57 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:41.852342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:41.852966 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:41.853250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:41.853530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:41.868371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.869569 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:41.871552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:41.888701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:41.889678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:41.890630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:41.890630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:41.890630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:41.890630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 359/691] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:41.898715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:41.939673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:41.954950 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85523a2-b119-4824-935a-03532f7ed2a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.024342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.025312 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.025596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.025867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.026663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:42.027102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.028663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.042856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 333/692] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:42.048410 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:42.066291 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffbb6-5be6-448e-936e-3e8ab276e731 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.137143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.138549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.138834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.139104 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.141324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:42.141628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.143005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.173428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 360/693] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:42.174681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:42.251393 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83389-05ae-4bff-8082-52a7593bf68c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.295756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:42.305596 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:42.320883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.321525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.321806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.322079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.332476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60437667-da9d-4100-89c0-d3aa10fca058 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.336262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.337720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:42.353208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:42.354091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:42.354971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:42.359856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 334/694] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:42.365596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:42.372876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.373786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.374262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.374727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.382988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.384724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:42.397829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:42.398776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:42.399761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:42.402235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 361/695] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:05:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:42.408574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:42.549178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09b28-7c4b-41d7-ac91-f285d30cbf27 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.582916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3c04f-9e9b-4898-9c6d-3505a0ff60a7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.618967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.619611 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.619888 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.620161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.620902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:42.621203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.622646 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.627614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:42.633994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.634651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.634884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.635157 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.639327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:42.639327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.641621 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.662117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 362/696] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:05:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:42.664514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 335/697] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:42.668106 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:42.670864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:42.749593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:42.807754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d809-bf51-4660-94cd-470bab3c5fc2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.809415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccebafb-4be3-41ca-8c02-95122345d751 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:42.844296 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.844907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.845187 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.845292 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef96d-a849-4317-8033-93e5f1925425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:42.845464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.845879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef96d-a849-4317-8033-93e5f1925425 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.846158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef96d-a849-4317-8033-93e5f1925425 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.846260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=cb269fa787664ebead0e2e9fd21ce219) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:42.846434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef96d-a849-4317-8033-93e5f1925425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.846525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.847914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce7a8f-0949-48e6-9348-fd1addec5b66 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:42.856439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 336/698] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:42 2022] GET /identity/v3/role_assignments?user.id=cb269fa787664ebead0e2e9fd21ce219&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:42.857713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425454-5b54-43ac-9405-b55983cfe393 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.857997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425454-5b54-43ac-9405-b55983cfe393 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.858268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425454-5b54-43ac-9405-b55983cfe393 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.860771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 337/699] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:42.868822 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:42.869153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:42.869363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:42.888742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:42.889918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:42.920188 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:42.932379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef96d-a849-4317-8033-93e5f1925425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:42.934185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef96d-a849-4317-8033-93e5f1925425 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:43.006066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 338/700] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:43.108000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 363/701] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 19:05:43.116403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.174169 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:43.217089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dff029-b9c5-4cb3-9528-a2d1acbb29b7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.217549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dff029-b9c5-4cb3-9528-a2d1acbb29b7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.218042 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dff029-b9c5-4cb3-9528-a2d1acbb29b7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.219722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 364/702] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:05:43.225963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.226354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.226730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.252851 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:43.254186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:43.256616 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da7d87-c50f-4f00-98ef-17436ff27072 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:43.300919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:43.302401 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.302854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.303315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.320650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:43.324941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:43.350909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:43.352034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:43.352908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:43.352908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:43.352908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:43.354852 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 339/703] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:43.364781 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.379454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 365/704] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:43.381732 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.427752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:43.508148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:43.521277 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72501-895d-4f0f-9c01-6d06d31a8725 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:43.558525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d628321-23ed-45ce-a2d8-c824bd8aa953 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:43.585556 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:43.586552 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.587025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.587545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.588427 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:43.588922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:43.590515 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:43.597815 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:43.599067 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.599414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.599684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.611571 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:43.614121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:43.616741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 340/705] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:43.622205 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:43.633284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:43.634239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:43.635177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:43.635177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:43.635177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:43.635177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 366/706] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:43.646706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.748519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:43.784978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db9855-a02c-4986-bb16-74b1acc8d2a6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:43.785245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181fdfd-db7b-4c16-9c35-5339fc8d6680 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:43.822625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:43.823414 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.823766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.824105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.829534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:43.830271 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:43.830690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:43.830969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:43.831239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:43.831638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:43.831945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:43.833071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:43.833636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:43.845692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:43.846615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:43.847545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:43.852150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 367/708] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:43.852365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 341/708] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:43.853056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.853232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:43.865683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:43.970248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6cd26-86d4-459b-8d42-4949c727c9e8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:43.991205 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0e53e-081c-4a59-99cf-5f5cc1434fdf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:44.008173 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.008716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.008994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.009384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.010071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.011597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.028642 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.029323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.029605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.029878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.030518 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.031906 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.034271 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:44.042833 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 342/709] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:05:43 2022] POST /identity/v3/users => generated 254 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:44.044917 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:44.045823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:44.047062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:44.048113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:44.059232 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 368/710] 10.222.0.18 () {56 vars in 1129 bytes} [Fri Sep 16 19:05:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:44.059232 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:44.149417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:44.181635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dae08-4aa8-4bd6-9956-0dd38c61b82a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:44.199783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2d76b-5177-4a9a-a076-35c9552b571e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:44.233818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.234774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.235202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.235686 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.236571 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:44.237099 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.238317 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.239665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.239897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffd945-891a-4d71-a049-8b767fa0b3da None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.240131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.240496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.241295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:44.241707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.243242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.247123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 369/711] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Sep 16 19:05:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:44.248727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66324a2-33e6-46aa-840d-2eebeda2f121 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.249162 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66324a2-33e6-46aa-840d-2eebeda2f121 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.249580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66324a2-33e6-46aa-840d-2eebeda2f121 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.251211 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 370/712] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:44.256508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.256953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.257372 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.262857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 343/713] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:44.263721 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:44.278919 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:44.280344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:44.355277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:44.367625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 371/714] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:44.368459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:44.395155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecc778-d193-4808-b8c3-b0781ec6f003 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:44.421126 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:44.433785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.434763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.434763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.435041 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.435713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:44.436014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.437387 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.444543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 344/715] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Sep 16 19:05:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:44.492513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774ae74-49bf-4228-9362-f1ebe7bf0e6b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:44.530098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.530944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.531411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.531829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.532781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=8d7573ee32f6443892a361f012c2f204) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:44.533258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.534878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.549780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 372/716] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:44 2022] GET /identity/v3/role_assignments?user.id=8d7573ee32f6443892a361f012c2f204&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:44.690860 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:44.807942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ee481-2fa7-42dd-9fa9-5094627b8971 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:44.846071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:44.846709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:44.846984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:44.847252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:44.847866 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:44.849328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:44.860670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:44.861715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:44.863402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 345/717] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:44 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:44.868369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:44.934024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:44.989639 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3bd70e-527c-47f3-8681-e503aceba4b0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:45.031024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:45.032026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:45.032507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:45.032978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:45.033805 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:45.034355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:45.036118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a39c1-e205-42b9-a8b1-ef04464e0f52 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:45.047345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 373/718] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:05:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:45.053058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:45.138730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:45.203883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0d09-5b90-4c91-9865-dd51b8bc36c9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:45.260021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:45.260635 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:45.260911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:45.261180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:45.261808 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] RBAC: Adding query filter params (type=network) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:45.262104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:45.263500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91facd-c851-4630-8ad1-c681bd510319 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:45.268887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 346/719] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:05:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:45.425409 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ac10-cd39-465b-acf8-d947d59c9787 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:45.426830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ac10-cd39-465b-acf8-d947d59c9787 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:45.426830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ac10-cd39-465b-acf8-d947d59c9787 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:45.427963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 374/720] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:45.433320 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:45.433604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:45.433874 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:45.464461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:45.465777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:45.552470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 347/721] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:45.918119 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ac10-cd39-465b-acf8-d947d59c9787 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:46.060965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ac10-cd39-465b-acf8-d947d59c9787 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:46.094289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:46.112513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:46.113471 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.113816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.114096 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.127605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:46.132971 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:46.133938 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:46.135208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 375/722] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:46.140437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:46.213286 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:46.268106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce2dc-c414-45de-9a45-b3991ba194e7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:46.310846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:46.311488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.311768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.312040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.312702 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:46.313001 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:46.314368 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:46.322877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 348/723] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:46.328364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:46.464091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86cef7-50cd-4f99-9766-a6c48327aaa8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:46.498516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:46.499142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.499451 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.499726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.510613 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:46.512076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:46.526574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:46.527634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:46.528987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:46.528987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:46.528987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:46.528987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 376/724] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:46.536447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:46.643927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c774ba-ea50-4859-b107-2eca267cfbf5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:46.653045 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:46.693584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:46.694290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.694570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.694846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.695521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:46.695828 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:46.697201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a406d-3ceb-49f4-a4a5-c09dbd6b6bfe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:46.713208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 349/725] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:46.719625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:46.761443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:46.843174 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e283e63-46eb-4b7e-9961-6dd6b9dc0412 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:46.854984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602cb2a-e2a0-4040-9245-cbf1833b11b4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.855270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602cb2a-e2a0-4040-9245-cbf1833b11b4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.855569 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602cb2a-e2a0-4040-9245-cbf1833b11b4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.856904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 350/726] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:46.862504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.862792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.863378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.883140 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca077a19-8159-4740-952a-9211986774cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:46.883814 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca077a19-8159-4740-952a-9211986774cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:46.884093 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca077a19-8159-4740-952a-9211986774cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:46.884367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca077a19-8159-4740-952a-9211986774cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:46.892445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca077a19-8159-4740-952a-9211986774cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:46.894047 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca077a19-8159-4740-952a-9211986774cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:46.903423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:46.904688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ca077a19-8159-4740-952a-9211986774cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:46.905832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:46.906742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:46.908347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 377/727] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:46.918284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca077a19-8159-4740-952a-9211986774cd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.035739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:47.044494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca077a19-8159-4740-952a-9211986774cd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:47.066233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 351/728] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:47.088867 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:47.089666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.090126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.090546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.091528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:47.091999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:47.093659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:47.112371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 378/729] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Sep 16 19:05:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:47.124396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.232708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:47.236548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a957a6-1cae-4dae-8c06-9884e6a3b4e5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:47.272185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ab48b-8d3c-4c65-9f13-a2a6d7ec4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:47.272802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab48b-8d3c-4c65-9f13-a2a6d7ec4c68 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.273274 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab48b-8d3c-4c65-9f13-a2a6d7ec4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.273544 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab48b-8d3c-4c65-9f13-a2a6d7ec4c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.350041 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab48b-8d3c-4c65-9f13-a2a6d7ec4c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:47.351488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab48b-8d3c-4c65-9f13-a2a6d7ec4c68 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:47.452286 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 352/730] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 19:05:47.452576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:47.458125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.459015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef6210-21db-434a-90e6-7838f86aaca8 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.459548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef6210-21db-434a-90e6-7838f86aaca8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.459994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef6210-21db-434a-90e6-7838f86aaca8 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.461748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 353/731] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:47.466782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b087-1aab-4c08-b0f3-82613ede5934 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.467252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b087-1aab-4c08-b0f3-82613ede5934 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.467732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b087-1aab-4c08-b0f3-82613ede5934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.485919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-be22b087-1aab-4c08-b0f3-82613ede5934 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:47.487406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-be22b087-1aab-4c08-b0f3-82613ede5934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:47.563226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 354/732] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 19:05:47.564818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29073654-c9bf-4a2d-aedb-99836735e616 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.565098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29073654-c9bf-4a2d-aedb-99836735e616 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.565367 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29073654-c9bf-4a2d-aedb-99836735e616 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.567255 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 355/733] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:47.567813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29073654-c9bf-4a2d-aedb-99836735e616 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.606327 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24682b-5810-4ba2-a930-edd759a64926 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:47.650880 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:47.652703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.653273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.654005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.669043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:47.671044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:47.671158 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29073654-c9bf-4a2d-aedb-99836735e616 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d9ce6e52-68db-4941-86c2-5f30fb5924b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:47.690980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:47.692217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:47.693115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:47.693115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:47.693115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:47.693115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 379/734] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:47.693368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.694138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.694138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.700724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:47.715430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:47.716590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:47.717870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:47.726305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:47.727830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.728317 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.729241 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.744279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:47.752263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:47.753171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:47.754015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:47.761299 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 356/735] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:47.761299 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.800962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 380/736] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:47.802295 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.898785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a7d9a-76a6-4a4a-b645-6a980f2047fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:47.912969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdafd-d067-406a-a9c7-e922f804b000 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:47.944439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:47.948050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:47.948985 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.949701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.950609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.950976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:47.951739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:47.952022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:47.952292 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:47.952483 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:47.952973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:47.953075 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:47.953281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:47.954636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:47.955039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:47.967347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 381/737] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:47.967347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:47.974636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 357/738] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:47.976467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.027643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.087701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3694-eb4b-48c7-b7f0-a0e42c14f7a6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.089113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7ef25-743e-4f7e-be2c-4589e4d62e11 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.125937 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.126885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.127446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.127890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.134676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.135527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.135904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.136287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.142984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.144553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.150278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:48.156500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:48.157431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:48.158298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 382/739] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:48.159005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:48.164421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:48.165380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:48.166304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:48.166304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:48.166304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:48.166304 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 358/740] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:48.166530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.224806 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.269497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.270612 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154873-b683-47bc-a6d5-4f7c3b302c20 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.272810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4b351-7042-441f-9887-00f9764e1cee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.303768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.304396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.304674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.304945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.306300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.306908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.307189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.307499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.308143 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.309538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.311508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.312941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.324993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 359/741] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:05:48 2022] POST /identity/v3/services => generated 251 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:48.325688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:48.327863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:48.328741 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:48.329672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:48.334304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 383/742] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:48.334304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.443711 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864cee8-b2ea-4bab-9f61-0dd5630f3bfc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.452018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42757e1-b7aa-4404-b569-1318e9daecee None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.468408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.481329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.481991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.482275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.482553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.483230 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:48.483565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.484984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e2abc-4bf4-430e-a594-c39b6da1ac94 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.493032 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.493644 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.493926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.494196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.494968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:48.495145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.496349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 360/743] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:48.496680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.497351 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894a63d-cb54-4d95-b083-71b1d13972a9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.497646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894a63d-cb54-4d95-b083-71b1d13972a9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.497924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894a63d-cb54-4d95-b083-71b1d13972a9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.503226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 361/744] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:48.503226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894a63d-cb54-4d95-b083-71b1d13972a9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.516772 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 384/745] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:48.517617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.524440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.614550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894a63d-cb54-4d95-b083-71b1d13972a9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.625726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a655-19ac-4b8f-a456-c43e465084fb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.649318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.649931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.650210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.650486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.651183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:48.651515 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.652952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81e47-fbba-49af-a170-738f9ef049f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.664846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.670378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 362/746] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:48.670605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.670605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.670605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.670799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.670799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.670799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.681638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.683094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.685400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:48.686603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:48.698355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:48.699532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:48.700635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:48.700635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:48.700635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:48.704256 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 385/747] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:48.705193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.735482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.760153 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 363/748] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 19:05:48.761480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.774818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:48.814999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c35f6-3a7b-4b9b-814c-472902b895a8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.853876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.854878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.855427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.855907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.865730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.867601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:48.880597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:48.881558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:05:48.882549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:48.889355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff6804-8c71-4c9f-a7f5-e24c40f22fc8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:48.889952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 386/749] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:48.890996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:48.933040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:48.934037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:48.934577 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:48.935101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:48.936064 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=cb269fa787664ebead0e2e9fd21ce219) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:48.936607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:48.938263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:48.944584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 364/750] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/role_assignments?user.id=cb269fa787664ebead0e2e9fd21ce219&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:48.945686 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.015372 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063dc94-15f6-4cee-a7cc-d7bbf8db1f76 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.019798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.088354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.088978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.089253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.089523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.090177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:49.090480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.092075 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.096504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.107760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 387/751] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:49.108621 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.140019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5aeb2-f1d1-4aa5-9b74-3148eb24e089 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.179789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.180508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.180782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.181178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.181905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:49.182207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.183651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.207906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 365/752] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Sep 16 19:05:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:49.208709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.222340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb8da-f0ea-4d2c-bcd0-f6a2ed9d240b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.260788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.261807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.262270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.262750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.263638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.265367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.287591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 388/753] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:05:49 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:49.288820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.313832 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.367389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27748b64-c8e9-48b9-956d-57043dec3ef5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.439264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.464085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.464746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.465026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.465298 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.481862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.483378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:49.502209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:49.503198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:49.504246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 366/754] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:05:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:49.510482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.518107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc690c-7eee-4739-ab59-c98416a3803d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.557974 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.558413 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.558731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.558963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.639434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926fbdf-3686-481f-aa00-a2d8ab30d4bd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.641064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.650432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.686523 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.687614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.687901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.688180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.688913 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:49.689585 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.691373 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.714145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 367/755] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:05:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:49.720994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.759611 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 389/756] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:05:49.771997 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:49.788035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.863977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ee193-46b5-4df2-9866-e3b967385391 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.928756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:49.929917 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:49.930703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:49.932018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:49.951752 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:49.954291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:49.968149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350d324-c574-45e6-a52f-df16f5a19dd4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:49.978576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:49.988484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:49.989962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:49.990914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:49.990914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:49.990914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:49.994113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 368/757] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:50.000012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.028979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.029942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.030443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.030915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.042722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.044476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:50.080322 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:50.081262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:50.082144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:50.082144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:50.082144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:50.082666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 390/758] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:50.092439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.205948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255bfc1-f7d1-402b-b823-3337828f7403 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:50.227974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:50.242326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b68f9-6fed-4152-a16e-1e027083ccf0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:50.268858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.269759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.270574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.270843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.271505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:50.271802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.273168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.290689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 391/759] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:50.292740 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.322848 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.323518 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.323806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.324084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.324271 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:50.324739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:50.325040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.326502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.346213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 369/760] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:50.351722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.478023 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e8f77-3420-4ee9-b617-d73aa1103ae1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:50.503392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c6ddc-7ad4-467f-925e-84d4b372e551 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:50.543886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.544654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.544940 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.545216 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.552984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.554638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:50.568596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:50.569570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:50.570463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:50.574662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 370/761] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:50.578646 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.580524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.581071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.581349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.581612 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.599258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.601513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:50.616107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:50.617086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:50.617985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:50.622196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 392/762] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:50.623961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.635649 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:50.711202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:50.736155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad03b-99d0-4887-a7a4-39d7d45c9a86 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:50.778317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.779308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.779740 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.780160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.780993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:50.781443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.781687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8fb3-449b-4382-9b7b-0110a446b2a6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:50.783321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95fbcc-2290-4a02-9d4f-c9742cd6a82e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.796191 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 393/763] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:50.797784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd208a0a-96ae-493f-b1d7-41b9d4ad1d1e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.798221 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd208a0a-96ae-493f-b1d7-41b9d4ad1d1e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.798671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd208a0a-96ae-493f-b1d7-41b9d4ad1d1e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.805125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 394/764] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:50.806630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.807064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.807735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.825235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:50.825850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:50.826125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:50.826396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:50.827079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:50.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:50.828078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:50.828834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:50.829587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:50.849615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 371/765] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:50.849615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.931314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 395/766] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:05:50.931314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:50.974149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:50.999698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:51.022946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c9e96-f1d6-42d7-8a70-3d16a7145a39 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:51.034675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed372-f257-4c4e-a04d-3dbc29edc86e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:51.080202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:51.080823 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.081100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.081372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.082100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=355f8f0689c54248b22dbeb1b578c77e) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:51.082435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:51.083912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:51.100845 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 372/767] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:50 2022] GET /identity/v3/role_assignments?user.id=355f8f0689c54248b22dbeb1b578c77e&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:51.112848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:51.114431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.114715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.114985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/8d7573ee32f6443892a361f012c2f204/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.120960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=8d7573ee32f6443892a361f012c2f204, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:51.122436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:51.147927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 396/768] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:50 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/8d7573ee32f6443892a361f012c2f204/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:51.269416 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:51.409586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468438d7-32ce-4827-a7db-e7604518ba03 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:51.451507 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:51.452516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.452991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.453467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.454245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:51.455862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-de999fdc-3812-46a9-8986-c2198a6d4751 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:51.466428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:51.467504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:51.468352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:51.471056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 373/769] 10.222.0.18 () {56 vars in 1129 bytes} [Fri Sep 16 19:05:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:51.477445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:51.589240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913e54b-c7c8-4095-811d-55af8cb9ca87 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.590331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913e54b-c7c8-4095-811d-55af8cb9ca87 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.590895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913e54b-c7c8-4095-811d-55af8cb9ca87 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.593484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 374/770] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:51.608563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.608861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.609116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.616109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da5a19-00ba-43e7-89f5-2ddcf2d9dec8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:51.639668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:51.646351 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:51.647603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:51.660053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:51.674009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:51.674627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.674909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.675183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.675849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:51.676364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:51.677978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:51.683461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 397/771] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Sep 16 19:05:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:51.689414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:51.790745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 375/772] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:05:51.853771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fb291-690b-49aa-a01b-1d46c21fceda None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:51.901570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:51.902541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:51.903038 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:51.903527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:51.904356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:51.904856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:51.906587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b162c-334d-42ca-b070-a6cada2e23ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:51.917783 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 398/773] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Sep 16 19:05:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:51.920279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.064004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce1a2-129a-4b86-8faa-a8af8f9e8ec2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.064632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce1a2-129a-4b86-8faa-a8af8f9e8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.065142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ce1a2-129a-4b86-8faa-a8af8f9e8ec2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.068851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b16e4-5b24-4305-b759-1acc3951d37f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.071488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 399/774] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:52.080615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.081129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.081597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.104453 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.105272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.105272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.105555 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.111370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] RBAC: Adding query filter params (interface=public, service_id=2b106193b4aa4f7d9f9b08a43fe19d5a, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:52.111370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:52.111370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:52.117452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 376/775] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:05:51 2022] GET /identity/v3/endpoints?service_id=2b106193b4aa4f7d9f9b08a43fe19d5a&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:52.119650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:52.122120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.122965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:52.123837 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:52.226953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:52.227146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0a86d-d1b9-41ed-aa85-96cf5fec405c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.243175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 400/776] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:05:52.248113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.282101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.282714 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.282993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.283267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.284388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:52.289069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:52.298983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 377/777] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:52.395641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea7c89-5f21-4893-a190-ad3df22b8760 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.445306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.446020 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.446301 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.446626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.463801 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:52.468819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:52.469765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:52.470846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 401/778] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:52.476058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.538873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.544462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:52.593128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9a8d-0a12-420e-8876-c403bce4cac1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.602896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:52.632628 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.633258 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.633533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.633812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.634591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:52.634903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:52.636459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:52.648784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 378/779] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:52.656588 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.674587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad72121-b69f-4308-b507-bed92297847c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.725155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.726116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.726399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.726676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.727551 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:52.729613 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:52.742371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:52.743417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:52.744265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 402/780] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:52.749987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.778940 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021ac75-01a8-4c07-8bea-0902534e0d9e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.806197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:52.819390 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.820254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.820602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.820941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.832973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:52.834519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:52.861138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:52.862107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:52.862996 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:52.862996 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:52.862996 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:52.871776 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 379/781] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:52.878708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:52.883967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954eb778-6b31-4a4b-8dca-8ea25840e3a9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:52.899417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:52.927744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:52.931310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:52.931310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:52.931310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:52.931310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:52.931310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:52.931310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:52.936556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 403/782] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:52.941941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:53.053151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee6d8-b5a9-4186-9496-21c1da8357b1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.089894 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.090503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.090891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.091165 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.091830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:53.092128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.093591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:53.103628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 404/783] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:53.104758 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:53.133424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1692f-2c6a-4b81-a55e-1af69d3e7a2f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.149400 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:53.212887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.214623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.215496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.216171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.217066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:53.217576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.217853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e2492-58b5-49d0-9ee8-7f90ac5ee4e4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.219830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:53.245995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 380/784] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:53.254193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:53.257440 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.258059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.258334 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.258605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.259310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] RBAC: Adding query filter params (interface=public, service_id=6e75553dc89e4052887c60cf0f5a9cff, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:53.259623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.263330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:53.274868 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 405/785] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:05:53 2022] GET /identity/v3/endpoints?service_id=6e75553dc89e4052887c60cf0f5a9cff&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:53.275730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:53.322941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:53.388520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a202060-860b-4298-8e72-46ae9d6bd4ac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.390267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:53.428847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.429738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.430156 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.430565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.431865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.433555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:53.443440 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 406/786] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:05:53 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:05:53.458359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191ecea-995c-4fd9-af6c-f9c4534f4245 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.525775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.526440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.526721 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.526997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.543405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.545498 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:53.563871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:53.564877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:53.566227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 381/787] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:53.571638 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:53.699669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0bf9f-ca18-4b01-8008-0d73402ae54f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.742014 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.742631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.742911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.743185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.743948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:53.744254 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.745670 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec23ca4-81f4-4649-879c-7e126b0fc4c4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:53.765544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 407/788] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Sep 16 19:05:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:53.771490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:53.791427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcc55e-8ed1-43dd-a9c3-5d4ef5e9621c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.791702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcc55e-8ed1-43dd-a9c3-5d4ef5e9621c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.791922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcc55e-8ed1-43dd-a9c3-5d4ef5e9621c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.794426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 408/789] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:53.798407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.798691 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.799871 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.821092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:53.823213 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:53.859209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:53.905759 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:53.916270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ba4a-37fe-44e9-ada2-6c32eef4eafd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:53.960349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 409/790] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:05:53.972762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:53.973494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:53.973773 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:53.974101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:53.984745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:53.986266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5667e6de-958c-4362-8ff6-f456423e1370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:54.001958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:54.002948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:54.003882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:54.003882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:54.003882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:54.003882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 382/791] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:54.012411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.028117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ef519-b0f5-4b4b-babc-377cd3f4b571 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.028411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ef519-b0f5-4b4b-babc-377cd3f4b571 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.028679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ef519-b0f5-4b4b-babc-377cd3f4b571 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.030271 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 383/792] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:05:54.035259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.035650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.035929 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.054605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:54.055762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:54.178949 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31691b9-38d6-46f9-86a5-ddb1a0ef502c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:54.200935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 384/793] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:05:54.226833 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:54.227475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.227761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.228037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.228687 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:54.228991 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:54.230393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:54.247455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 410/794] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:54.248763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.300902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.331335 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:54.387619 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:54.411997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d61e9-906f-4b1d-9018-4d781d057d98 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:54.449025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:54.450101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.450585 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.451058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.460166 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:54.461891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:54.482964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:54.483948 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:54.484862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:54.490869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 385/795] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:54.495555 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.501937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c3411-12c6-4755-82d4-92b8b0bda9a7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:54.553806 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:54.555769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.556477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.557714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.580123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:54.586745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:54.587933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:05:54.588778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:54.591385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 411/796] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:54.592380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.647967 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce05b76-c32a-483f-86f8-364830a3ca1f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:54.696135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:54.696775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.697058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.697335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.698044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:54.698354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:54.699838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:54.708861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:54.724332 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 386/797] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:54.725351 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.797788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:54.825661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5c4b-e23f-435c-a6e9-8f9953804ed8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:54.862671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:54.863888 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.864358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.864811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.870838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38936e3b-4a78-48d8-a4b1-1db0dc08e0da None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:54.876345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:54.881891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:54.882816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:54.883702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:54.885955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 412/798] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:54.886905 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:54.918256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:54.918893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:54.919170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:54.919493 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:54.920282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:54.920587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:54.922106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:54.932228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 387/799] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:54.937862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.002844 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.007314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150eca8d-1a91-451c-8ae1-29ee6140b7c3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.046143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.047091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.047485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e20f2-b44c-4cba-8b0d-77c9fa636d2f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.047779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.048236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/cb269fa787664ebead0e2e9fd21ce219/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.051051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=cb269fa787664ebead0e2e9fd21ce219, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.053244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.055677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.070249 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 413/800] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:54 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/cb269fa787664ebead0e2e9fd21ce219/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:55.071706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.102357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.103679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.104251 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.104545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.105302 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:55.105889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.108258 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88201f3c-bd5a-47f5-aa63-b993dcd5b870 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.120797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 388/801] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:55.122743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a4e0-32d8-4a59-9424-86593b90536a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.123040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a4e0-32d8-4a59-9424-86593b90536a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.123340 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a4e0-32d8-4a59-9424-86593b90536a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.125739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 389/802] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:55.127492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7a4e0-32d8-4a59-9424-86593b90536a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.211817 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92eca1-3d92-4396-a964-3f3aa2b98288 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.252875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7a4e0-32d8-4a59-9424-86593b90536a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.256261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.256898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.257180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.257456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.270363 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.272092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.278118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f7d50323-460a-4fb6-ab8c-cec40831632a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:55.288853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:55.289820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:55.290701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:55.290701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:55.290701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:55.290701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 414/803] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:55.291100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.292395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.292395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.294825 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.294975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.295307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.295837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.308820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.310558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.312193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:55.313461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:55.326746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:55.327809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:55.328687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:55.328687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:55.328687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:55.328687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 390/804] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:55.329556 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.364622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.397008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 415/805] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:05:55.397848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.485207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d21ee6-bc28-44f4-b205-aa245e8785c9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.540311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.540923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.541199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.541467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.542107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:55.542404 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.543798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.558012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc26b4-5871-4d8d-bf29-025ebd2942df None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.629747 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.630374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.630645 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.630911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.631579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:55.631874 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.633355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.655409 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 416/806] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:55.661309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.711767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.745724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 391/807] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:55.746579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.782568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.788019 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e155a-ebcb-40a8-a6a8-730c6aa29beb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.828433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.829065 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.829345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.829621 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.837197 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.838684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:55.850857 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:55.852040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:05:55.853278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:55.856724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 417/808] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:55.857573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.862455 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5504-c57d-4cbc-9164-3594a52b4ae1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:55.904336 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:55.905240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:55.905575 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:55.905791 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:55.906378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:55.908338 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:55.920174 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:55.922270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:55.923602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 392/809] 10.222.0.18 () {56 vars in 1129 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:55.928780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:55.968240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:55.976738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93bff3-cfc0-4ae3-9933-d4e4d4d6dfc4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.025370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.027484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.027968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.028435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.039090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.041218 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.049132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.055347 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bf9a1-34b0-44d5-bc0d-1c4d8f494fe6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:56.056559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:56.057474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:05:56.058454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:56.061241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 418/810] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:56.062173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.103897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.104845 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.105225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.105601 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.106508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:56.106923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.108838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.131028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 393/811] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:05:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:56.132276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.183660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6146-103c-4c5a-9b90-08ac5190e10e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.220451 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.221083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.221327 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.221568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.222183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:56.222591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.224618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62f292-31d7-4baf-9c34-2d9021e4dab3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.233690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 419/812] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:56.233690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd10154-7cd0-47c4-b298-376359cc99cd None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.233690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd10154-7cd0-47c4-b298-376359cc99cd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.233690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd10154-7cd0-47c4-b298-376359cc99cd None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.233690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 420/813] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:56.233690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd10154-7cd0-47c4-b298-376359cc99cd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.239047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53a588-8052-4e18-907d-1532b5864377 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.256112 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.275133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.275800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.276078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.276350 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.277065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:56.277369 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.278775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.295733 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 394/814] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:56.296347 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.310890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.356427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd10154-7cd0-47c4-b298-376359cc99cd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.392468 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c3b0-3fc7-4696-9433-57249544b5a5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.399170 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.400196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.400481 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.400756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.426871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.427510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.428210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.428500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.429132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:56.429435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.430792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1488-90c7-4bb4-a1d5-fe74b5d2d63d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.436042 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 395/815] 10.222.0.18 () {56 vars in 1124 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:56.439463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.439463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.439463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.457867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:56.459060 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:56.482360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.483726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.507584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.529115 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 396/816] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:05:56.539738 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.547310 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.619805 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 421/817] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:05:56.620926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.638170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5118f-cbb2-4568-ac3a-edf6cd08d49c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.673003 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.673632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.673903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.674172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.684915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.686393 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:56.701534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:56.702533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:56.703516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:56.703516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:56.703516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:56.703516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 397/818] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:56.708331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.761066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29860-459e-464f-8eb3-b40caf777570 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.774432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.783516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:56.798802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.799446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.799722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.799992 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.800712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.802602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.833204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d0aa-7fc6-40b9-87cc-bc99ca3abefa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.835787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 422/819] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:05:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:56.837149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.888018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.888754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.889178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.889546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.902219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.903967 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:56.921289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:56.922863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:56.923809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:56.923809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:56.923809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:56.927264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 398/820] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:56.928358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:56.952475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f5c6-f402-4843-aa58-e243223a6ad7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:56.990157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:56.990769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:56.991048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:56.994172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:56.994172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:56.994172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:56.994172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.012494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 423/821] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:57.013559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.048997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c2440-1df6-4aff-90ab-a651bd19eb04 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.058057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.065518 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.094475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.095125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.095441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.095716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] PATH_INFO: `/v3/services/network` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.096303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.097669 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:57.107237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:57.108748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:57.109843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 399/822] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:56 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:57.110628 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.139163 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6cb77-71c4-4e35-9526-922b885b87fa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.178759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.179465 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.179748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.180024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.180669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:57.180975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.182341 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.201040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 424/823] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:57.201941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.237654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96555dde-bd38-4c99-9b93-3f542c642466 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.274476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.275502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.275984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.276454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.283787 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.286076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:57.297939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:57.298836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:57.299775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:57.302225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 400/824] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:57.303220 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.307896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.315457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d996-0ea1-44d7-ad6c-c27d6cdc2242 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.340659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.358419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.359219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.359536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.359821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.360685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:57.360995 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.362635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.373572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 425/825] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:57.374480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.420900 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5c9c-5a36-421a-b15a-7c6355d72c5f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.456656 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.457842 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.458142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.458409 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.465240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.466674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:05:57.478437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:57.479578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:57.480492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 401/826] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:57.480976 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.495678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8107b-e158-4aff-adff-e5554ced5a94 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.530986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.531627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.531909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.532199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.532864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:57.533173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.534586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138724f-2966-4d96-b8df-485902434a20 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.552727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 426/827] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:57.552727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf734-3a76-4d88-ba1b-eda0f6f575d2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.552727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf734-3a76-4d88-ba1b-eda0f6f575d2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.552727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf734-3a76-4d88-ba1b-eda0f6f575d2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.552727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 427/828] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:05:57.552727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf734-3a76-4d88-ba1b-eda0f6f575d2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.575970 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.588657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.597896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d0d1d-f8f4-4764-a897-afe68e040e66 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.635593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.636201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.636478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.636748 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.637373 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] RBAC: Adding query filter params (type=network) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:57.637672 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.639025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.649126 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 402/829] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:57.649126 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.671062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf734-3a76-4d88-ba1b-eda0f6f575d2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.711833 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.712435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.712708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.712975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.713768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:57.714070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.715610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a3b52-6660-416a-a9e3-cfc309935d0c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.733456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 428/830] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:57.734747 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.735036 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.735328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.750165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df6d1d-f74f-4468-a129-595b6a54e8fc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:57.754907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:57.756134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:57.786809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:57.787804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:57.788079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:57.788405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/355f8f0689c54248b22dbeb1b578c77e/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:57.804535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=355f8f0689c54248b22dbeb1b578c77e, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:57.806019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:57.820716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 403/831] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:05:57 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/355f8f0689c54248b22dbeb1b578c77e/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:05:57.821774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.830854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.843321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 429/832] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:05:57.843321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:57.859132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:57.962353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df61827-afe7-4746-9a41-4bebca14f9b8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:58.005500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:58.006104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:58.006388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:58.006665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:58.007363 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:58.009182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:58.040622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 404/833] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:05:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:05:58.050431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca943fd2-d4c8-4e2f-8cc8-50429e35beed None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:58.119437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:58.120091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:58.120436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:58.120717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:58.121513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=8d7573ee32f6443892a361f012c2f204) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:58.121826 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:58.123342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:58.131427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 430/834] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:05:57 2022] GET /identity/v3/role_assignments?user.id=8d7573ee32f6443892a361f012c2f204&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:58.213668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:58.302656 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:58.361683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f261d-d12a-4dd2-a354-2489cab92394 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:58.399412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:58.400048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:58.400325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:58.400587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:58.411144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:05:58.416421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:58.417383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:58.418250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 405/835] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:58.423654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:58.494997 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:58.528431 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c2807-477d-4f01-b6c9-1ff333aaab4a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:58.564768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:58.565297 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:58.565576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:58.565848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:58.566509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:58.567373 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:58.568735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:58.578089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 431/836] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:05:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:58.583816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:58.689761 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800ff9-9b52-4617-8530-62c03631b0b7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:58.715731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:58.728699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:58.729344 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:58.729621 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:58.729893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:58.742075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:58.743553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:58.758953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:58.759978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:58.760856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:58.760856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:58.760856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:58.760856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 406/837] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:58.767362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:58.873145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d4a3b-3f47-4786-bc3d-1ee7bf3e5aea None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:58.905052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:58.911144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:58.912124 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:58.912598 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:58.913063 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:58.913949 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:58.914460 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:58.916088 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:58.928936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 432/838] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:58.936451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.049706 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c1fc-610c-4ef0-91b9-27067fda769c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:59.098226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:59.098868 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.099152 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.099458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.107201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:59.108735 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d510472e-6229-47f6-9f4c-b185110bbc38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:05:59.120998 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:59.121952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:59.122887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 407/839] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:05:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:59.129793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.160883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87543d-d81a-48b5-8de7-fc1d8633c862 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.161204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87543d-d81a-48b5-8de7-fc1d8633c862 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.161433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87543d-d81a-48b5-8de7-fc1d8633c862 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.163027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 408/840] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:05:59.168582 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.168582 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.168582 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.187757 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:05:59.189237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:05:59.221290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:59.240442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596e8b5-a85f-424b-990f-502fe0122eff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:59.275353 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:59.275953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.276229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.276497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.277214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:59.277516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:59.278979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:59.279207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 409/841] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:05:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:05:59.302276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 433/842] 10.222.0.18 () {56 vars in 1135 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:59.308194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.411964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61dbe8-87a6-4d5c-a49d-6a4790b37423 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:59.446762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:59.447398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.447682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.447956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.482362 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:59.520311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:59.521756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:59.545862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.599527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:59.646396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 410/843] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:05:59.647580 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.662604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e482d-fe20-46c0-8826-6a8c2beda990 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:59.710002 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:59.710844 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.711123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.711426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.712018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:59.713950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:59.723609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:59.727856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:59.729044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:05:59.729884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 434/844] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:05:59.736538 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.770955 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415c127-b25c-4a6e-9152-81cb086f82a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:59.811797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:59.812736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.813021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.813298 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.827409 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:59.828922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:05:59.850056 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:05:59.851543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:05:59.852608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:05:59.852608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:05:59.852608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:05:59.852608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 411/845] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:05:59.858180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.858485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b48ec-3064-47fc-8f88-34a80a17a3bb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:05:59.876699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:05:59.907450 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:05:59.908642 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:05:59.909048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:05:59.909684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:05:59.910576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:05:59.911363 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:05:59.913663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:05:59.920223 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 435/846] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:05:59.927205 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:05:59.981997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d3a1-e4bc-416e-9411-5d315c8b1437 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:00.006120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:00.032587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:00.033212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.034034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.034367 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.035314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:00.035691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:00.037750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:00.046443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c570a1-ad6c-44fa-8323-e446c0202794 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:00.054391 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 412/847] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:00.059973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:00.090319 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:00.091040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.091494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.091830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.092511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:00.092880 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:00.094298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:00.106364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 436/848] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:05:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:00.180755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a0ab1-d944-440c-b9e5-b82ed88bd303 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:00.226248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:00.226914 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.227199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.227503 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.235713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:00.237225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-73c6c74d-bbfc-40bd-b641-27ad4b370483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:00.251146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:00.252104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:00.253178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 413/849] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:00.258764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:00.309959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:00.332539 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a891d-8a38-4b82-875d-79ded5d7d0f8 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.332879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a891d-8a38-4b82-875d-79ded5d7d0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.333142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a891d-8a38-4b82-875d-79ded5d7d0f8 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.334684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 414/850] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:00.339956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fda44-1d83-49d0-a74e-92fd0f5181fe None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.340237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fda44-1d83-49d0-a74e-92fd0f5181fe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.340516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fda44-1d83-49d0-a74e-92fd0f5181fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.359552 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-926fda44-1d83-49d0-a74e-92fd0f5181fe None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:00.360816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-926fda44-1d83-49d0-a74e-92fd0f5181fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:00.371269 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037c820-ca91-44eb-b6a6-434172edef7c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:00.396425 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:00.416087 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:00.416983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.417626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.417902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.418588 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:00.419200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:00.420936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:00.436734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 437/851] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:00.442482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:00.452830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 415/852] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:00.547514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec8ecd-2987-40f3-a28b-b5091cc0e327 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:00.548961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:00.583602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:00.584228 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.584555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.584826 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.585565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=cb269fa787664ebead0e2e9fd21ce219) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:00.585873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:00.587348 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828370d-3abd-4820-b5e3-a099fbc83988 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:00.594511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 438/853] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:00 2022] GET /identity/v3/role_assignments?user.id=cb269fa787664ebead0e2e9fd21ce219&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:00.717259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d60814-243e-4faf-8c56-ce37fdf44e82 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.717436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d60814-243e-4faf-8c56-ce37fdf44e82 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.717723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d60814-243e-4faf-8c56-ce37fdf44e82 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.719115 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 416/854] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:00.725944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.726396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.726766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.732824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d60814-243e-4faf-8c56-ce37fdf44e82 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:00.747149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:00.748578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:00.827483 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 439/855] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:00.857584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d60814-243e-4faf-8c56-ce37fdf44e82 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:00.893895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:00.894579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:00.895028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:00.895337 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:00.906116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:00.911678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:00.912587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:00.913431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 417/856] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:00.918985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.017073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:01.023413 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4787395-71ea-4a21-a152-fd8fb2d3c599 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.065921 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.067358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.067792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.068207 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.069009 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:01.069476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.070985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.080209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 440/857] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:01.092637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.106796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.127242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:01.218615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf1e0-b97a-4f8f-acbd-f2535cdfa04c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.223231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4dde5-3333-45c3-a203-4f793d49cbf9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.254820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.255851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.256334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.256802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.264241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.264873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.265149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.265419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.266448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.267893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.270731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.272485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:01.278432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:01.279398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:01.280326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 441/858] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:01.287566 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:01.293961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:01.294910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:01.296141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:01.296141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:01.296141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:01.297927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 418/859] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:01.303829 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.314493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:01.399926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:01.414261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466e90-01a5-4981-930c-e3b8d93aee31 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.422399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613e53-ef36-4442-a9e6-83fb099ec42b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.456872 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.457895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.458005 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.458471 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.458577 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.458909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.459257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.459728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.460081 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:01.460438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.461382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:01.461921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.462283 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.464434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.474268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 442/860] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:01.474495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 419/861] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:01.478263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.479768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.584121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b277f36-6b12-43b8-ad67-1c5fa4549866 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.592606 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:01.613344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c2cee-39e8-4f53-9aaa-47c776480f02 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.623982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.625007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.625456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.625947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.626791 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:01.627318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.628938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.634934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 420/862] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:01.651408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.652069 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.652334 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.652602 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.663490 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.664967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:01.678404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:01.679458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:01.680426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 443/863] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:01.685321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.688493 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:01.801510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793b0a-019b-4f5b-8a58-7881c03ec6ba None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:01.839636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:01.840177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:01.840455 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:01.840729 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:01.841464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:01.841772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:01.843226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:01.862175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 421/864] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:01.868206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:01.971423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfa886-a68a-4068-a6b3-d6adcb07965c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:02.007051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c793b-6a70-46c6-a6d5-41495d6c9dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:02.007759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c793b-6a70-46c6-a6d5-41495d6c9dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.008035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c793b-6a70-46c6-a6d5-41495d6c9dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.008305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c793b-6a70-46c6-a6d5-41495d6c9dc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.024979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:02.079655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c793b-6a70-46c6-a6d5-41495d6c9dc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:02.081220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c793b-6a70-46c6-a6d5-41495d6c9dc1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:02.183509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 444/865] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:02.189075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:02.254200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:02.299830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566346c-a26f-4d65-a694-8c3292a97a6d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:02.334932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:02.335599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.335876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.336150 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.346728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:02.348751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:02.356875 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262156e1-6315-4f95-9178-f19c889d1e76 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.357345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262156e1-6315-4f95-9178-f19c889d1e76 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.357422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262156e1-6315-4f95-9178-f19c889d1e76 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.360259 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 445/866] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:02.364269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:02.365189 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:02.366088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:02.366088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:02.366088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:02.366258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.366258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.366258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.366446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 422/867] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:02.372528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:02.388998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:02.390320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:02.440007 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:02.474493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 446/868] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:02.494929 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119256-354e-4330-8098-dcb6f2914b88 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:02.508682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:02.529988 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:02.530599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.530887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.531160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.531860 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:02.532165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:02.533572 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:02.546469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 423/869] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:02.552066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:02.659385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed21536-2e4b-4c48-9cc3-05405755ee78 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:02.700435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:02.701390 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.701901 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.702362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.710526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:02.712524 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:02.724534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:02.725423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:02.726302 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:02.728625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 447/870] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:02.736585 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:02.775766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:02.814536 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:02.848888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0bdcc-ab47-4fe9-b64e-7a43b2dc941f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:02.895491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a07645-d419-4bbb-802b-f45f963ce394 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:02.899973 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:02.900837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.901312 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.901785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.902663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:02.903550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:02.905036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:02.921811 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 424/871] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:02.928979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:02.941088 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf99d6-2f8f-4336-a23d-a6a86f1c3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:02.942263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf99d6-2f8f-4336-a23d-a6a86f1c3b98 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.942549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf99d6-2f8f-4336-a23d-a6a86f1c3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.942826 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf99d6-2f8f-4336-a23d-a6a86f1c3b98 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.943527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf99d6-2f8f-4336-a23d-a6a86f1c3b98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:02.945340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf99d6-2f8f-4336-a23d-a6a86f1c3b98 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:02.966204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 448/872] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:02 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:06:02.967499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116143-3c13-4a5a-b96b-b14ad252d130 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.967780 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116143-3c13-4a5a-b96b-b14ad252d130 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.968050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116143-3c13-4a5a-b96b-b14ad252d130 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.969414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 449/873] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:02.975188 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:02.975570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:02.975757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:02.981876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:03.000801 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:03.001924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:03.066467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9184847-dfb9-4a57-9132-5256a018cb09 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:03.095987 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 450/874] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:03.115278 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:03.116322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:03.116835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:03.117306 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:03.118252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=355f8f0689c54248b22dbeb1b578c77e) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:03.118771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:03.120459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:03.128258 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 425/875] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:02 2022] GET /identity/v3/role_assignments?user.id=355f8f0689c54248b22dbeb1b578c77e&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:03.367848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:03.393139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:03.474680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f95f94-808d-4d9e-9ae0-4839aa9979db None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:03.521214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:03.521862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:03.522145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:03.522423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:03.523008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:03.524417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:03.533570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:03.534632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:03.535756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 451/876] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:03.543084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:03.643735 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:03.649628 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4d49-debd-44e0-af34-c72016729d9d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:03.686255 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:03.686869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:03.687148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:03.687550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:03.688181 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:03.688482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:03.689847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe725d-ee91-4346-a83d-2cddf49c251b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:03.695008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 426/877] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:03.700786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:03.794046 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:03.811265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41b97-a28c-49fb-8426-25c8188f5ca0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:03.846290 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:03.847315 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:03.848386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:03.848876 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:03.849711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:03.850226 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:03.851827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77b7b-5bca-4271-8abb-1677e801d547 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:03.863853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 452/878] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:03.996549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be892f8-eda9-440a-8458-03dcfa227f54 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:03.996928 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be892f8-eda9-440a-8458-03dcfa227f54 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:03.997113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be892f8-eda9-440a-8458-03dcfa227f54 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:03.998706 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 427/879] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:04.005285 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee72ad-c124-4c94-a2d2-d63a53b862f6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:04.005619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee72ad-c124-4c94-a2d2-d63a53b862f6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:04.005842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee72ad-c124-4c94-a2d2-d63a53b862f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:04.025054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-32ee72ad-c124-4c94-a2d2-d63a53b862f6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:04.026262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-32ee72ad-c124-4c94-a2d2-d63a53b862f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:04.101493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 453/880] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:04.369525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be892f8-eda9-440a-8458-03dcfa227f54 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:04.474379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be892f8-eda9-440a-8458-03dcfa227f54 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:04.509014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f7e2b-7c6b-4793-85c4-4ef2e2a59419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:04.509875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7e2b-7c6b-4793-85c4-4ef2e2a59419 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:04.510326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7e2b-7c6b-4793-85c4-4ef2e2a59419 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:04.510754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7e2b-7c6b-4793-85c4-4ef2e2a59419 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:04.511589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f7e2b-7c6b-4793-85c4-4ef2e2a59419 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:04.513105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f7e2b-7c6b-4793-85c4-4ef2e2a59419 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:04.530282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 428/881] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:04 2022] POST /identity/v3/services => generated 269 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:05.215708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8153d-286f-4d83-ab94-cdcada401ef2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:05.216246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8153d-286f-4d83-ab94-cdcada401ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:05.216246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8153d-286f-4d83-ab94-cdcada401ef2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:05.217787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 454/882] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:05.223188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd0e56-8a8b-435e-b968-61649d46b39e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:05.223661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd0e56-8a8b-435e-b968-61649d46b39e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:05.223749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd0e56-8a8b-435e-b968-61649d46b39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:05.246977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4bbd0e56-8a8b-435e-b968-61649d46b39e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:05.248567 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4bbd0e56-8a8b-435e-b968-61649d46b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:05.264256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:05.330764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 429/883] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:05.461753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c868e40-e62c-446c-b411-8965cebe4d60 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:05.462406 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c868e40-e62c-446c-b411-8965cebe4d60 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:05.462896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c868e40-e62c-446c-b411-8965cebe4d60 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:05.464743 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 455/884] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:05.470193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:05.470494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:05.470776 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:05.488950 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:05.490121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:05.557394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 430/885] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:05.609704 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c868e40-e62c-446c-b411-8965cebe4d60 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:05.692141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:05.713752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c868e40-e62c-446c-b411-8965cebe4d60 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:05.750066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:05.751060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:05.751590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:05.752081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:05.752869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:05.754555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:05.764081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:05.765115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:06:05.765969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:05.768716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 456/886] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:06:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:05.779361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:05.857999 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:05.872070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:05.892746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbb19c-7348-4e8b-a833-99a86cde62cc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:05.939793 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:05.940722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:05.941130 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:05.941548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:05.943038 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:05.943522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:05.945510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:05.958012 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 431/887] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:06:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:05.958887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:05.979314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ea31c-a4e0-43f7-98ce-b88e12409a48 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.014555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.015553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.016138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.016609 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.027364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:06.032869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:06.033840 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:06.034707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:06.037895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 457/888] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Sep 16 19:06:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:06.044382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.060714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:06.081098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd372-dad6-4208-ad40-000d93d47682 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.125417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.125999 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.126675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.126965 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.127637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:06.128098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.130368 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.142004 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 432/889] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:06:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:06.143847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.147510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:06.182598 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574db5c-47c9-41cd-a5e7-a57d39a64ca0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.221253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.222054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.222358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.223544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.224210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:06.224528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.225937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc2e8c-1834-4f37-99ca-3fe16b4556c2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.243881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 458/890] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:06.243881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321bc8-d405-404e-bf00-ac35380e8c05 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.243881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321bc8-d405-404e-bf00-ac35380e8c05 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.243881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321bc8-d405-404e-bf00-ac35380e8c05 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.244443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 459/891] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:06.246221 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3321bc8-d405-404e-bf00-ac35380e8c05 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.261046 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833559d-cf0a-4315-ac15-0468fb439b3c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.300613 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.301546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.302219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.302382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.303468 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] RBAC: Adding query filter params (interface=public, service_id=84fa6bc31ec249c88fbcbc3b37b534ac, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:06.303938 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.306120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f5d58-6c6b-4699-9772-0178d2081af9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.319780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 433/892] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/endpoints?service_id=84fa6bc31ec249c88fbcbc3b37b534ac&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:06.321319 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.321744 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.322162 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.331133 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:06.346386 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:06.347597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:06.372559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3321bc8-d405-404e-bf00-ac35380e8c05 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.408034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.408582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.408854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.409123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.415137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:06.419960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.421384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.423484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 434/893] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:06.424448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:06.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:06.446055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:06.446988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:06.446988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:06.446988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:06.449235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 460/894] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:06.454785 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.555094 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf8c6-e5df-4404-bc5c-09791f87e2ad None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.568575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf77a3-d027-435b-a2ea-e64b08685b4e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.588416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:06.608228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.608999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.609507 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.609634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.609869 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.610144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.610267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.610787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:06.610906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.611092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.611717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.612514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.613237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.623308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 435/895] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 19:06:06.628004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 461/896] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:06.634804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.711692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:06.743078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.759814 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3735f9-1f44-4fc6-bfa0-9f3c8c293dc6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.800507 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.801214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.801480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.801749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.809642 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.814330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3461b8be-6c91-4acf-b5bc-e36e57b9cc34 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:06.826203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:06.827418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 436/897] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ad3fe-6429-41d0-a393-57bdffb128d1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.828730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ad3fe-6429-41d0-a393-57bdffb128d1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.829603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ad3fe-6429-41d0-a393-57bdffb128d1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.830248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 437/898] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:06.836018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.836296 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.836563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.856834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:06.858010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:06.862403 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174907a6-c5a3-4997-aa1b-7f45fcd67f3d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:06.902002 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:06.902630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:06.902915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:06.903172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:06.903848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:06.905279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:06.927758 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 462/899] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:06 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:06:06.929006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:06.950219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 438/900] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:06.977138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.056339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4cb1-f5f2-41d3-96b4-7a8cdc51576e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.099125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.100101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.100514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.100937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.101692 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:07.101993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.103447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.123199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 463/901] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Sep 16 19:06:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:07.128891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.180908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.234270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.236484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce60aa-dd4e-474f-90c8-2f1931827fc6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.278037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.279607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.280092 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.280671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.353890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.355809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.370476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b592242-abc5-4b8d-baeb-e54604e5290f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.384778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.409471 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.410153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.410498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.410866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.411442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.412324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.414032 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:07.424795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:07.426520 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:07.427652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:07.430114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 464/902] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:07.438710 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.474836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 439/903] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:07.480188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.561385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0dd1d-eff6-45ad-b566-0675cb5157e7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.607901 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.608561 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.609064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.609341 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.609972 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:07.610331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.612422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.615869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da185daf-998b-4278-9008-1d14837a75fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.620044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 465/904] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:07.624494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.656817 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.659830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.661402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.661849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.662284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.675140 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.677765 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.688818 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:07.698554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:07.699732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:07.700801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:07.700801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:07.700801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:07.707652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 440/905] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:07.708794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.755453 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663bfe9-da0d-4a90-aeb3-6ddf40c6e1c8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.799180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.799979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.800350 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.801049 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.801773 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:07.802222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.804418 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.811122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 466/906] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:07.812226 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4227c6-0b40-4867-a08d-8d043b86a454 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.816438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.855593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.856228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.856511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.856790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.857456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:07.857878 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.859571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:07.875218 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 441/907] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:07.881092 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:07.930373 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.945222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716e155-2ead-403e-9270-d3c1ad55e6da None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:07.963408 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:07.989180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:07.990398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:07.990886 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:07.991435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:07.992460 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] RBAC: Adding query filter params (interface=public, service_id=885b9dc189524f53b701b606a35d789b, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:07.993000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:07.994703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:08.001750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 467/908] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/endpoints?service_id=885b9dc189524f53b701b606a35d789b&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:08.007444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:08.017606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59027f-11b9-41a3-9ff1-149697592496 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:08.055387 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:08.056361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:08.056637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:08.056905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:08.075225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:08.076815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:08.089674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:08.090764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:08.091853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:08.093524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 442/909] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:08.098847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:08.131279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537bf7-78f1-4f52-8475-a60eeeebe915 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:08.167889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:08.168564 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:08.168842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:08.169115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:08.169779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:08.171042 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:08.183328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 468/910] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:06:08 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 19:06:08.229295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4607475-14b3-4211-8abf-510b182a5564 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:08.251251 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:08.267509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:08.268219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:08.268566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:08.268907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:08.270187 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:08.270571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:08.272114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc39b3f-5291-4231-a5a6-cc2a29ba4c0e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:08.286311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 443/911] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:08.292461 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:08.317201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:08.400724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b4287-cf53-4529-939a-39447e01fcb9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:08.436185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:08.436893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:08.437235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:08.437570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:08.438357 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] RBAC: Adding query filter params (role.id=6ec1c3ef130347d4bbfdb77a7776246c, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=355f8f0689c54248b22dbeb1b578c77e) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:08.438731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:08.440220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fd15d-edba-4404-a1ab-3944afbe36be None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:08.446686 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 469/912] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:08 2022] GET /identity/v3/role_assignments?user.id=355f8f0689c54248b22dbeb1b578c77e&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6ec1c3ef130347d4bbfdb77a7776246c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:08.963205 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706bc6b-9acd-4acd-bf18-1480e0e01cbc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:08.963538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706bc6b-9acd-4acd-bf18-1480e0e01cbc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:08.963952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706bc6b-9acd-4acd-bf18-1480e0e01cbc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:08.965634 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 444/913] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:08.971028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384890-7605-4d09-9ff7-c9d3ebcdff5d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:08.971408 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384890-7605-4d09-9ff7-c9d3ebcdff5d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:08.971753 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384890-7605-4d09-9ff7-c9d3ebcdff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:08.990724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ae384890-7605-4d09-9ff7-c9d3ebcdff5d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:08.991939 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ae384890-7605-4d09-9ff7-c9d3ebcdff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:09.079395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 470/914] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:09.166488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d647a-1101-482b-93e0-77406609fde0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.166845 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d647a-1101-482b-93e0-77406609fde0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.167022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d647a-1101-482b-93e0-77406609fde0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.168668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 445/915] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:09.174070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.174589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.175086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.194288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:09.195802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:09.198917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:09.274595 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 471/916] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:09.351511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d647a-1101-482b-93e0-77406609fde0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.456109 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d647a-1101-482b-93e0-77406609fde0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:09.492138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:09.493022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.493446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.493881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.494602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:09.496126 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:09.505231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:09.506394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 19:06:09.507438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:09.509673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 446/917] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:06:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:09.515682 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.551445 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.601850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:09.624293 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82421d1a-6a81-4ad3-b266-b2231cee18d8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:09.661086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:09.661686 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.661961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.662227 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.662508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccdfb0-daf9-4e56-bf9a-bf87b7727c66 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:09.662849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:09.663146 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:09.664649 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:09.669878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 472/918] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:06:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:09.675278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.679555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:09.700181 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:09.701110 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.701592 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.702062 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.702892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:09.705232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:09.716131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:09.717147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:09.717982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:09.721738 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 447/919] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:09.728276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.796033 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27adab03-fceb-45aa-8fc6-367de91bc3b8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:09.833121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:09.834062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.834566 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.835123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.836078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:09.836600 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:09.838257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:09.842258 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42456f5-38f1-47d5-857c-2a5b08e4ccf3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:09.844228 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 473/920] 10.222.0.18 () {56 vars in 1114 bytes} [Fri Sep 16 19:06:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:09.850204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.855685 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:09.887617 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:09.889365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:09.889796 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:09.890266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:09.891040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:09.891581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:09.893083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:09.904736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 448/921] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:09.908625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:09.928243 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:09.966034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7970c-2c8d-4f38-9ad7-3d0aca75e467 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:10.004500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:10.005101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.005375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.005640 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.006310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:10.008165 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:10.034570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdbc28-5c11-4fe3-a5e4-c9c3e8b37a09 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:10.037310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 474/922] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:06:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:10.092335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:10.092867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.093140 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.093464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.094089 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:10.094386 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:10.095851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:10.102481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 449/923] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:10.108714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:10.232279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799197b-1a72-4163-aa94-04ad61ee69e9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:10.258670 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:10.272276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:10.272890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.273169 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.273439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.274135 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] RBAC: Adding query filter params (interface=public, service_id=7f3881436c124137baa14683c851db15, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:10.274438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:10.275841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a0830-9ee8-4e7b-8c8f-e65f93b4aaf4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:10.281749 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 475/924] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:06:10 2022] GET /identity/v3/endpoints?service_id=7f3881436c124137baa14683c851db15&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:10.287741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:10.322627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:10.399786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aada0-1d76-4e93-a8b6-579c9ee8dcec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:10.434927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6fa94-2be3-4b3c-a63b-edb38932c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:10.435562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6fa94-2be3-4b3c-a63b-edb38932c8fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.435838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6fa94-2be3-4b3c-a63b-edb38932c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.436106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6fa94-2be3-4b3c-a63b-edb38932c8fb None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.436707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6fa94-2be3-4b3c-a63b-edb38932c8fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:10.438055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6fa94-2be3-4b3c-a63b-edb38932c8fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:10.444614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 450/925] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:06:10 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 19:06:10.493220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2ca0-e379-47f0-bc41-32c86a154cc5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.493544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2ca0-e379-47f0-bc41-32c86a154cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.493772 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2ca0-e379-47f0-bc41-32c86a154cc5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.495424 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 476/926] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:10.501172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d48ed-b185-43e4-9bb7-050d43524b2f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.501464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d48ed-b185-43e4-9bb7-050d43524b2f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.501735 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d48ed-b185-43e4-9bb7-050d43524b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.521593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-646d48ed-b185-43e4-9bb7-050d43524b2f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:10.522817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-646d48ed-b185-43e4-9bb7-050d43524b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:10.597659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 451/927] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:10.706883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddf91-1635-41bd-8e6f-810620309785 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.707495 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddf91-1635-41bd-8e6f-810620309785 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.707590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddf91-1635-41bd-8e6f-810620309785 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.709371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 477/928] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:10.714674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:10.714952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:10.715260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:10.734083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:10.735250 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:10.756190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:10.803262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 452/929] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:10.871754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ddf91-1635-41bd-8e6f-810620309785 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:10.979975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ddf91-1635-41bd-8e6f-810620309785 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.014620 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.015250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.015562 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.015833 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.016411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.017761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:11.027193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:11.028141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:11.029608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 478/930] 10.222.0.18 () {56 vars in 1132 bytes} [Fri Sep 16 19:06:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:11.036670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.085907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.142104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb13b9-7091-4c48-a5cc-68b0a1f51cd2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.155639 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:11.179547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.180398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.180817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.181254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.182017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:11.182466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.183984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.189237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 453/931] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:11.194480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.217104 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d99b5-e920-4f22-a347-d5242bf5524d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.219920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:11.258632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.260274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.261000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.261659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.273666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:11.279028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:11.279989 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:11.280816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:11.283528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 479/932] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:11.289550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.306240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe88df-2b4f-4ad1-b4b8-93a526c53c5c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.342129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.342737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.343017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.343302 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.343935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:11.344236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.345583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.356248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 454/933] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:11.357995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.418062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b54f-16c9-488e-b666-868703c96876 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.435743 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:11.461469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.462457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.462965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.463465 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.464319 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:11.464921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:11.466389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.468356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.479412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 480/934] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:11.485222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.486593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f5077-0578-414d-8cae-760a3d9008e8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.535101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.537049 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.537577 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.537995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.539060 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.541325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.576500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 455/935] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:06:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:11.611040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46efda-2d3a-45ac-a8d2-461a72bca482 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.659321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.660022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.660885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.661185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.673509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.675081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:11.691138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:11.692210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:11.693084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:11.693084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:11.693084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:11.693084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 481/936] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:11.698675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.813293 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379e40-4e82-41ac-9796-2b802bc2308c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:11.848713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:11.849325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:11.849601 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:11.849873 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:11.850509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:11.850807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:11.852191 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:11.861851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:11.866447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 456/937] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:11.872501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:11.898204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:11.982925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538723-9f23-492a-9d49-5e865da5f11a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:12.018690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:12.020101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.020527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.020945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.028963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:12.030584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:12.041976 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:12.043070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:12.044103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:12.044931 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 482/938] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:06:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:12.050477 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:12.156905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1815f8-21a4-42b1-a1ae-41ff3e50ecc3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:12.193874 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:12.194484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.194764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.195039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.195799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:12.196111 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:12.197539 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b53095-56af-4fc2-ae47-287b4e4d5357 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:12.219725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 457/939] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Sep 16 19:06:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:12.225723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:12.316177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:12.342068 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3204dfb-7b7a-48dd-ac53-e47c35b8e97f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:12.377040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:12.377689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.377963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.378237 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.389387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:12.390828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:12.409717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:12.411246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:12.412467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:12.412467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:12.412467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:12.412637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bdd5e-a335-4c8e-aed6-f49602948008 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.412637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bdd5e-a335-4c8e-aed6-f49602948008 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.412637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bdd5e-a335-4c8e-aed6-f49602948008 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.413704 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 458/940] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:12.414809 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 483/941] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:12.419895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.420349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:12.420645 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.421114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.444638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:12.446681 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:12.516867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:12.533802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 459/942] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:12.543387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e0ae3-1eb1-43de-bd37-f5a98612bcc2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:12.573774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:12.574469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.574752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.575030 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.575819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:12.576129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:12.577667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:12.591681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:12.595465 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 484/943] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:12.597047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:12.706168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102002c-1663-4594-8ac6-f82589cfaaf6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:12.751799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:12.752447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.752731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.753010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.761231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:12.762763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-cc5accc0-2552-41f1-83e1-277e6cd6f093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:12.774436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:12.775321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:12.776413 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 460/944] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:12.781726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:12.806237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147461cd-d6c4-46d2-9eca-824279ec69a6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.806535 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147461cd-d6c4-46d2-9eca-824279ec69a6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.807280 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147461cd-d6c4-46d2-9eca-824279ec69a6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.808964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 461/945] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:12.813978 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.814493 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.814983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.833055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:12.834231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:12.885354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:12.892195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63452802-c5cf-4ef8-90f9-fbf8a9b8bbe7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:12.911403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 462/946] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:12.911403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:12.942177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:12.943087 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:12.943530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:12.944108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:12.945280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:12.945992 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:12.947593 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:12.961735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 485/947] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:12.967745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.032549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:13.041845 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3b1b8-99db-46c1-9686-8e3c83375c6c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.087647 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.088477 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.088932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.089365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.090148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.091763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.094546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING keystone.server.flask.common [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 16 19:06:13.100658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91c89-93d8-4585-97e5-3bc839877a53 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.112594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 19:06:13.112594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 16 19:06:13.112594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [parameters: {'id': '8711838c99e14f718e80e9bc6a23d96b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 16 19:06:13.112594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8711838c99e14f718e80e9bc6a23d96b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8711838c99e14f718e80e9bc6a23d96b. Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 19:06:13.122596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 16 19:06:13.123643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 16 19:06:13.124817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 19:06:13.125906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 19:06:13.127002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 19:06:13.127993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'id': '8711838c99e14f718e80e9bc6a23d96b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.129098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:13.129892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8711838c99e14f718e80e9bc6a23d96b. Sep 16 19:06:13.130932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.138224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 463/948] 10.222.0.18 () {60 vars in 1140 bytes} [Fri Sep 16 19:06:12 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:13.139907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.147680 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.148462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.148831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.149198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/355f8f0689c54248b22dbeb1b578c77e/roles/6ec1c3ef130347d4bbfdb77a7776246c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.152004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=355f8f0689c54248b22dbeb1b578c77e, role_id=6ec1c3ef130347d4bbfdb77a7776246c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.154007 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.172246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 486/949] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:06:12 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/355f8f0689c54248b22dbeb1b578c77e/roles/6ec1c3ef130347d4bbfdb77a7776246c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:06:13.184802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:13.204734 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.282356 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f079ce-7f25-4a1d-980a-21b8a08d9187 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.331778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8bd20-6569-46f6-aaff-03abfdc5782e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.334379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.335951 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.336230 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.336521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.348200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.350908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:13.352083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:13.371750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:13.372724 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:13.374284 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 464/950] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:13.379740 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.385979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.386925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.387530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.388008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.388854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.391061 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:13.403250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:13.404287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:13.405116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:13.409017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 487/951] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:13.414488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.487513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:13.496630 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1b98c-80a1-4951-9a60-01f9ca930233 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.518659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166233-82e4-4a7a-9ae8-6d011abb4b40 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.532760 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.534032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.534522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.535005 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.536625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:13.536949 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.538395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.559118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.559961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.560240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.560551 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.561017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 465/952] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:13.561184 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:13.561544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.562913 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.572668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 488/953] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:13.578595 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.701965 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd852-56c9-48ec-af63-1b521f70bf68 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.741134 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.741754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.742030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.742302 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.742939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:13.743242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.744742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154012f0-13df-41f3-9230-e4b823e5c504 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.750475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 466/954] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:13.756525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:13.814937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:13.878829 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f42d8-450a-4c86-80e4-8ebc68f16f6d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:13.904796 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d0280-30ec-4293-867f-e2bca6eab4dd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.905090 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d0280-30ec-4293-867f-e2bca6eab4dd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.905342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d0280-30ec-4293-867f-e2bca6eab4dd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.906923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 467/955] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:13.912451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.912733 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.913008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.918298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:13.919715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:13.920170 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:13.920709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:13.921381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:13.922858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:13.932942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:13.934591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:13.947194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 489/956] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:06:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:13.951358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:14.027088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 468/957] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:14.305960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:14.409742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:14.414460 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805399cd-de38-431a-9571-98640ba9c07f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:14.450293 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:14.450943 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:14.451222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:14.451523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:14.452113 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:14.453474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:14.462993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:14.464137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:14.465017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 490/958] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:14.473287 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:14.573076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072f902-e1ba-4b09-a3f9-177aa77a2ae3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:14.606529 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:14.607136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:14.607445 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:14.607718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:14.608340 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:14.608639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:14.610002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d777b-1ec6-476d-ab8e-8379d4790d5c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:14.615372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 469/959] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:14.620683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:14.729912 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce4b76-18dd-42c6-8978-3b0ccd17a416 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:14.764282 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:14.765154 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:14.765605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:14.766085 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:14.766872 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:14.767374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:14.768898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3ec86-20de-46d1-9c71-286947ad397d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:14.774323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 491/960] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:15.469227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72291f3-0471-4449-9109-e7c031e15407 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:15.469576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72291f3-0471-4449-9109-e7c031e15407 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:15.469682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72291f3-0471-4449-9109-e7c031e15407 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:15.471160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 470/961] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:15.476636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0bac-5252-4956-adee-dc6baa44246a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:15.477069 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0bac-5252-4956-adee-dc6baa44246a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:15.477164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0bac-5252-4956-adee-dc6baa44246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:15.496079 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c68e0bac-5252-4956-adee-dc6baa44246a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:15.497253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c68e0bac-5252-4956-adee-dc6baa44246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:15.521247 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:15.569523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 492/962] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:15.845000 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72291f3-0471-4449-9109-e7c031e15407 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:15.857348 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:15.959155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72291f3-0471-4449-9109-e7c031e15407 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:15.964980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b573ed-67d6-4f3a-abc2-e98622fa21c9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:15.965600 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b573ed-67d6-4f3a-abc2-e98622fa21c9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:15.966132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b573ed-67d6-4f3a-abc2-e98622fa21c9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:15.967998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 493/963] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:15.974119 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:15.974691 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:15.975183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:15.995688 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:15.995982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:15.996536 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:15.996813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:15.997083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:15.998074 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:16.008085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:16.012961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:16.013941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.014789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 471/964] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Sep 16 19:06:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:16.022837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.103407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 494/965] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:16.117455 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:16.151389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8df2-3e05-4f57-a585-86ce5fadc8e9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.186054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.186663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.186940 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.187215 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.187909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:16.188209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.189563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ef1b1-eceb-45de-841a-cf5e145ba85d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.198349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 472/966] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:16.204397 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.207898 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390a4b9-002f-44e4-938e-9d1064aaf97b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.208182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390a4b9-002f-44e4-938e-9d1064aaf97b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.208456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390a4b9-002f-44e4-938e-9d1064aaf97b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.209857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 473/967] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:16.214517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.214798 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.215075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.234330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:16.235538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:16.304999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:16.322055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce0db-99ed-4109-9d16-1dfc58f08126 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.331499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 474/968] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:16.362991 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.364584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.364866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.365145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.377470 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.379010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.393719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:16.395369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:16.396296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:16.397558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:16.397558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:16.397558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:16.397558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 495/969] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:16.402423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.477024 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:16.521941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce090e8-b12b-4565-94cd-96a275bfd009 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.522135 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f640a2a-b844-4cc6-ab04-c7991da0b9e2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.557932 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.558544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.559006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.559309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.559959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:16.560066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.560257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.561037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.561505 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.561605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.562072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.563036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.564762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.569342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:16.579422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 496/970] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:16.580519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.583256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 19:06:16.583256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 16 19:06:16.583256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [parameters: {'user_id': 'cac250987e12462fbbb2928fadcf91a4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 16 19:06:16.583256 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 16 19:06:16.590916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 19:06:16.591997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 19:06:16.593121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 19:06:16.594330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 19:06:16.595551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 19:06:16.596696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cac250987e12462fbbb2928fadcf91a4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:16.603415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 16 19:06:16.604492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 475/971] 10.222.0.18 () {60 vars in 1131 bytes} [Fri Sep 16 19:06:16 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:16.605766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.693376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595e845-b2b0-466c-9585-ade5acdc8e33 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.715829 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9c605-2a10-41d9-abf8-d7eb413802cb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.731105 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.731916 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.732198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.732474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.739749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:16.740194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.741733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:16.753637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:16.755333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 19:06:16.756494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:16.757137 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.757461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.757819 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.758175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.759018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.760159 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 497/972] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:16.760761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.761183 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:16.771566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:16.772664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:16.773622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:16.780549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 476/973] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:16.780549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.824036 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:16.875405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa6f7-59e4-4bc5-bfd6-d0ee44efe923 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.893716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4d14-5639-4b60-b3c1-79cf13a5799b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:16.911141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.911901 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.912249 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.912584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.920607 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.922157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.934935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:16.935654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:16.936003 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:16.936342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:16.937141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:16.937377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:16.938422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 19:06:16.939578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:16.940309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:16.941116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:16.946744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 498/974] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:16.947842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:16.966251 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 477/975] 10.222.0.18 () {56 vars in 1149 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:16.967200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.012464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.047585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f5a69-f6ce-4deb-8112-240cc34a08c5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.065771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.066500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc60edc-41d8-45d8-a057-521641d04ee8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.086041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.086732 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.087017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.087307 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.088385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:17.088701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.090062 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e9f97-48cd-46c4-87cf-1d506949535d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.094649 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 499/976] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:17.096037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f3846-ad37-417b-a1e5-ec4d685361f1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.096384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f3846-ad37-417b-a1e5-ec4d685361f1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.097030 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f3846-ad37-417b-a1e5-ec4d685361f1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.098768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 500/977] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:17.099522 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f3846-ad37-417b-a1e5-ec4d685361f1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.105396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.106050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.106328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.106601 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.107310 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:17.107629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.109047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.145030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 478/978] 10.222.0.18 () {56 vars in 1101 bytes} [Fri Sep 16 19:06:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:17.145850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.231230 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f3846-ad37-417b-a1e5-ec4d685361f1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.257495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcd1c8-d16a-4812-988c-34d9e22f6a79 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.259430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.271537 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef763423-1786-40d7-b217-832dded32813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.272168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef763423-1786-40d7-b217-832dded32813 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.272451 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef763423-1786-40d7-b217-832dded32813 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.272771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef763423-1786-40d7-b217-832dded32813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.297178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.297816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.298101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.298347 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.299029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:17.299280 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.300695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06939497-8551-43e3-bb64-bea50f7da9c4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.306564 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 479/979] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:17.307924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.308211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.308488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.329784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:17.332025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:17.334233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.358760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef763423-1786-40d7-b217-832dded32813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.360314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef763423-1786-40d7-b217-832dded32813 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.415379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 480/980] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:17.416378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.468093 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 501/981] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:17.473715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef763423-1786-40d7-b217-832dded32813 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.519243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.526160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcbe5a-9b59-4d48-9839-387736ace3fc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.557983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.562395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.563134 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.563545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.563923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.564804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] RBAC: Adding query filter params (interface=admin, service_id=7f3881436c124137baa14683c851db15, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:17.565130 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.567238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.577462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 481/982] 10.222.0.18 () {56 vars in 1249 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/endpoints?service_id=7f3881436c124137baa14683c851db15&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:17.578629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.581768 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef763423-1786-40d7-b217-832dded32813 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.621078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.621700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.621974 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.622245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.633566 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.635006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:17.649110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:17.650136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:17.651052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:17.651052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:17.651052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:17.651052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 502/983] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:17.655848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.681594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a0148-8f38-4707-b81b-02b54a112f55 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.716596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.717196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.717472 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.717743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.718336 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.719727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.725561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 482/984] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 19:06:17.726323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.756525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.759186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fec65-5245-49b7-8ef0-2e20a6c954ab None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.802432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.802962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.803238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.803539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.804186 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:17.804486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.805861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.807722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:17.823306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 503/985] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:17.828889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:17.848990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4e6ca-a2e1-47aa-96b8-21d1607b0310 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.888089 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.888834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.889575 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.889851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:17.890504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:17.891941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:17.912802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 483/986] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:17 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:17.945398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb01ac-8329-4bf7-ad45-223514728244 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:17.992360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:17.993012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:17.993284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:17.993549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:18.001174 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:18.002699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:18.015582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:18.016700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:18.017831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 504/987] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:18.024695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:18.126819 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:18.145583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d9d81-4aba-4190-9596-1c13502f0891 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:18.179336 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:18.179947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:18.180221 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:18.180490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:18.181182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:18.181482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:18.182891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf9c8b-2fed-4521-b3fe-6077065eaa55 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:18.199432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 484/988] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:18.202013 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:18.254469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:18.307733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff30a6-8472-4c99-bf3a-e9043618263d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:18.341058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:18.341722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:18.342103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:18.342476 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:18.343311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] RBAC: Adding query filter params (role.id=6ec1c3ef130347d4bbfdb77a7776246c, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=355f8f0689c54248b22dbeb1b578c77e) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:18.343731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:18.345236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bad78-5b7b-45d7-b0c6-28f3381d95ee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:18.351819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 505/989] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:18 2022] GET /identity/v3/role_assignments?user.id=355f8f0689c54248b22dbeb1b578c77e&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6ec1c3ef130347d4bbfdb77a7776246c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:19.966595 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7e43-889a-4375-903f-a0f7b2e05f5f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:19.967149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7e43-889a-4375-903f-a0f7b2e05f5f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:19.967149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7e43-889a-4375-903f-a0f7b2e05f5f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:19.968551 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 485/990] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:19.973668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff6f4-0475-49c3-9b89-3b3c4c9a407a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:19.974045 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff6f4-0475-49c3-9b89-3b3c4c9a407a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:19.974221 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ff6f4-0475-49c3-9b89-3b3c4c9a407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:19.993191 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8d4ff6f4-0475-49c3-9b89-3b3c4c9a407a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:19.994393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8d4ff6f4-0475-49c3-9b89-3b3c4c9a407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:20.075618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 506/991] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:20.124688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f1c8-5efe-49ce-aca9-d0a79b12c57b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.124982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f1c8-5efe-49ce-aca9-d0a79b12c57b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.125253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f1c8-5efe-49ce-aca9-d0a79b12c57b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.126732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 486/992] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:20.131943 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.132235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.132513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.151112 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:20.152333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:20.193626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:20.221417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 507/993] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:20.338329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f1c8-5efe-49ce-aca9-d0a79b12c57b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.438863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f1c8-5efe-49ce-aca9-d0a79b12c57b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:20.476569 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:20.477129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.477475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.477745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.478332 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:20.479796 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:20.487005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:20.489984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:20.490960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:20.491849 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 487/994] 10.222.0.18 () {56 vars in 1126 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:20.496854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.508746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:20.593979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebf137-1965-4677-888f-c47f0de63277 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:20.600163 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f87afd-e431-41b1-b338-e247d366c96e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:20.626444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:20.627071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.627369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.627637 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.628211 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:20.630065 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:20.633476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:20.634069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.634345 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.634612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.635224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:20.635546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:20.636873 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7335b7a9-3220-437b-98ab-d6e6dba25fbc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:20.639029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:20.640100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:20.640948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 508/995] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:20.641855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b06cc-0bbc-4ed0-b3df-2f28aae4d88d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.641855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b06cc-0bbc-4ed0-b3df-2f28aae4d88d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.641855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b06cc-0bbc-4ed0-b3df-2f28aae4d88d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.643222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 509/996] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:20.645834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 488/997] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:20.646641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b06cc-0bbc-4ed0-b3df-2f28aae4d88d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.648364 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.687654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:20.719080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:20.747903 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b06cc-0bbc-4ed0-b3df-2f28aae4d88d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:20.749519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa37db-0c41-4247-b0dc-b2e7c6fb5aee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:20.780391 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:20.780984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.781256 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.781530 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.782139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:20.782282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:20.782437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:20.782830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.783107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.783407 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.783821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee4d05-f355-47b6-8507-a5178cdfd349 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:20.784025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:20.784324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:20.785655 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:20.788494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 510/998] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:20.790545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 489/999] 10.222.0.18 () {56 vars in 1122 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:20.794320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.794610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.794885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.795990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.813129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:20.814299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:20.880817 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 511/1000] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:20.881621 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:20.901531 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb80b8-6c51-4edb-8dc6-e411cb03d05c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:20.915744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:20.938745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:20.939375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:20.939657 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:20.939942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:20.940546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:20.940851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:20.942309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:20.944937 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:20.953870 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 490/1001] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:20.954675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:21.001338 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bc2c8-a821-4fcf-8156-320cc3012aa1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:21.039907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:21.040508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:21.040786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:21.041058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:21.041714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:21.043090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:21.060547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a1ccc-b2f3-4d80-9e21-457dc21d708f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:21.075356 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 512/1002] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:06:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:21.108513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:21.109131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:21.109412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:21.109684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:21.110388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] RBAC: Adding query filter params (interface=public, service_id=84e0f06037bf45eeb5878cedc1c8d852, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:21.110700 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:21.112103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:21.123749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 491/1003] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:06:20 2022] GET /identity/v3/endpoints?service_id=84e0f06037bf45eeb5878cedc1c8d852&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:21.125263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:21.152555 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:21.224959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:21.233936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:21.246591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbabaa-07eb-49c4-bc94-e960d504a709 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:21.275141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed25fd2-e2a5-42b5-90b1-d727fce2b290 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:21.286462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:21.287075 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:21.287447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:21.287727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:21.288341 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:21.289707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:21.297514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 513/1004] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:06:21 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 19:06:21.314827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:21.315540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:21.315827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:21.316097 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] PATH_INFO: `/v3/services/image` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:21.317032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:21.319209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:21.329182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:21.330264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:21.331105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 492/1005] 10.222.0.18 () {56 vars in 1105 bytes} [Fri Sep 16 19:06:21 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:21.338346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:21.447746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2e64-2ad5-434c-9a57-2bb7943994f1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:21.483695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:21.484301 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:21.484575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:21.484843 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:21.485462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] RBAC: Adding query filter params (name=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:21.485760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:21.487109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524ab1f-1a23-4f39-bc78-3ab5eba99b94 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:21.492480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 514/1006] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:21.497964 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:21.601268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9abcf-04cd-4940-aae0-7a9c55855815 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:21.626697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:21.638064 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:21.638674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:21.638951 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:21.639220 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:21.639872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:21.640199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:21.641554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1868401-b928-4991-a532-a91ffcea69f7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:21.646709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 493/1007] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:23.246154 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755417e-40cb-467e-b407-2e75179035b1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.246694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755417e-40cb-467e-b407-2e75179035b1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.246694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755417e-40cb-467e-b407-2e75179035b1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.248235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 515/1008] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:23.253424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f27dec-1c87-470c-a5eb-db03a83caaef None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.253875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f27dec-1c87-470c-a5eb-db03a83caaef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.254082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f27dec-1c87-470c-a5eb-db03a83caaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.272402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-69f27dec-1c87-470c-a5eb-db03a83caaef None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:23.273592 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-69f27dec-1c87-470c-a5eb-db03a83caaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:23.339676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 494/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:23.509668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44805c2c-cecb-41d0-9834-8ae41eadc798 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.510086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44805c2c-cecb-41d0-9834-8ae41eadc798 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.510255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44805c2c-cecb-41d0-9834-8ae41eadc798 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.511628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 516/1010] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:23.516943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.517244 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.517527 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.536119 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:23.537307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:23.603993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 495/1011] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:23.615539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44805c2c-cecb-41d0-9834-8ae41eadc798 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:23.637852 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:23.725139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44805c2c-cecb-41d0-9834-8ae41eadc798 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:23.758934 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447e7ed-1e03-449b-8a2b-3236c6814933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:23.759605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447e7ed-1e03-449b-8a2b-3236c6814933 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.759888 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447e7ed-1e03-449b-8a2b-3236c6814933 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.760162 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447e7ed-1e03-449b-8a2b-3236c6814933 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.761286 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447e7ed-1e03-449b-8a2b-3236c6814933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:23.762714 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447e7ed-1e03-449b-8a2b-3236c6814933 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:23.764457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 517/1012] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:23.770575 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:23.804842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1d4f0-a9ad-44a6-8ed2-6ab5120dd384 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.805192 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1d4f0-a9ad-44a6-8ed2-6ab5120dd384 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.805396 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1d4f0-a9ad-44a6-8ed2-6ab5120dd384 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.806715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 518/1013] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:23.811616 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.811897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.812167 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.820667 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:23.830516 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:23.831713 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:23.877639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af7b98-52d5-470d-ae9a-df1e9e535da0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:23.900845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 519/1014] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:23.901673 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:23.916941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:23.917562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:23.917851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:23.918128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:23.918784 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:23.920263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:23.924674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:23.948743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 496/1015] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:23 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:24.019359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715c6cc-779c-40da-9dfe-ab719424bd8f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:24.057628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:24.058272 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:24.058547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:24.058820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:24.059455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:24.062853 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:24.073759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:24.074866 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:24.076404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 520/1016] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:24.080882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:24.178889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:24.186026 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:24.188887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20606b85-658c-4cc7-8eb6-9788f0cf2a72 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:24.225984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:24.226619 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:24.226897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:24.227170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:24.227821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:24.228121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:24.229478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:24.234575 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 497/1017] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:24.241268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:24.275599 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:24.295716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4f073-ef24-45a4-9e83-32d09df6c50d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:24.331819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:24.332604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:24.332884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:24.333161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:24.333825 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:24.335487 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:24.345904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf4313-17f4-4060-bdee-798695894d3f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:24.360723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 521/1018] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:24 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:06:24.394107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:24.395305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:24.395626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:24.395856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:24.396494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:24.396790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:24.398334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669872-d02e-4b29-858c-accb002cebb1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:24.407330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 498/1019] 10.222.0.18 () {56 vars in 1112 bytes} [Fri Sep 16 19:06:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:24.410373 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:24.523161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dd886-94ef-4592-aca2-7d1b0ac84dbd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:24.561259 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b98b5-83bf-4296-8f5f-4e6966ff29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:24.561870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b98b5-83bf-4296-8f5f-4e6966ff29fd None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:24.562150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b98b5-83bf-4296-8f5f-4e6966ff29fd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:24.562452 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b98b5-83bf-4296-8f5f-4e6966ff29fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:24.563122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b98b5-83bf-4296-8f5f-4e6966ff29fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:24.564610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b98b5-83bf-4296-8f5f-4e6966ff29fd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:24.579983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:24.588353 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 522/1020] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:06:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:26.171303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3606f9-7a1d-4599-a7dc-015e4a25b4bb None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.171812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3606f9-7a1d-4599-a7dc-015e4a25b4bb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.171916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3606f9-7a1d-4599-a7dc-015e4a25b4bb None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.173273 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 499/1021] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:26.178915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f23b9-3840-4f89-b849-a5991180e95a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.179339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f23b9-3840-4f89-b849-a5991180e95a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.179529 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f23b9-3840-4f89-b849-a5991180e95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.198295 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9b1f23b9-3840-4f89-b849-a5991180e95a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:26.199587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9b1f23b9-3840-4f89-b849-a5991180e95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:26.279330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 523/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:26.532279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49921f23-506c-44f2-ae3c-de48beccc1a0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.532641 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49921f23-506c-44f2-ae3c-de48beccc1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.532827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49921f23-506c-44f2-ae3c-de48beccc1a0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.534209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 500/1023] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:26.539960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.540245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.540517 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.544307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49921f23-506c-44f2-ae3c-de48beccc1a0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:26.561411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:26.562585 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:26.572240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:26.637111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 524/1024] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:26.658804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49921f23-506c-44f2-ae3c-de48beccc1a0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:26.693731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ba529-c368-4bb8-b414-466ce94259d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:26.694360 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba529-c368-4bb8-b414-466ce94259d6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.694639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba529-c368-4bb8-b414-466ce94259d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.694912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ba529-c368-4bb8-b414-466ce94259d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.696048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ba529-c368-4bb8-b414-466ce94259d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:26.697476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ba529-c368-4bb8-b414-466ce94259d6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:26.699243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 501/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:26.711480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:26.771963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:26.816917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3b3d3-86f1-4b1f-b5e7-207a0c4f9e0f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:26.829674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb50a3-5227-4f3b-95f2-e12cecf0df67 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.829963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb50a3-5227-4f3b-95f2-e12cecf0df67 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.830243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb50a3-5227-4f3b-95f2-e12cecf0df67 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.831744 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 502/1026] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:26.836859 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.837141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.837415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.851333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:26.851916 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:26.852196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:26.852467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:26.853121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:26.854532 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:26.854967 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:26.856160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:26.877830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 525/1027] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:06:26 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:06:26.914156 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:26.941135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 503/1028] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:27.027793 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f81f-a30e-4047-984e-7131494da176 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.039595 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.069106 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.069301 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.069521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.071332 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.071332 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.071782 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:27.084678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:27.085842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:27.086994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 526/1029] 10.222.0.18 () {56 vars in 1105 bytes} [Fri Sep 16 19:06:26 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:27.091790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.119135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.196792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c18341-3a4d-4d92-87cc-73a664e26644 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.214367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.232540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.233151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.233426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.233695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.234321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:27.234620 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.236000 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.246380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 504/1030] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:27.248058 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.320887 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92930-b383-4c4d-8841-d62492f75f42 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.337705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.355103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308632b7-d470-4fe1-9171-ede050be6df0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.356279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.356913 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.357199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.357474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.369873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:27.374813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:27.375810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:27.376839 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:27.378738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 527/1031] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:27.385169 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.397152 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.397688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.397966 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.398237 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.398858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] RBAC: Adding query filter params (type=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:27.399361 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.400545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.410571 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 505/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:27.412291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.428130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.504248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4aab0-7ab9-4fed-a2b2-d5bfbdd0cab1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.524343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ae97f-59a1-42f9-84b4-b25e95c3c164 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.540512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.541118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.541394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.541663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.542378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:27.542679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.544060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.558171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 528/1033] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:27.559493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.561122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.561993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.562380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.563475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.564194 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] RBAC: Adding query filter params (interface=public, service_id=46023dba0a57468285c7e061aad4542d, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:27.564505 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.565857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.575797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 506/1034] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/endpoints?service_id=46023dba0a57468285c7e061aad4542d&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:27.576699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.592596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.665744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.672795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3827c-30d2-44de-ba01-7796503b1866 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.680965 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecd593-d5c8-49a8-976e-1ec5a3fae0c4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.709701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.710358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.710640 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.710911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.717335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.717952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.718223 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.718497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.719101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.720499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.721828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.723341 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.735316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 507/1035] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:27.745726 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:27.746779 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:27.747789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:27.747789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:27.747789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:27.747789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 529/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:27.753652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:27.838030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:27.862178 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18129d-c2d5-4bb8-86c8-fc9a29b3b81c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:27.900844 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:27.901648 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:27.902029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:27.902401 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:27.903159 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:27.903593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:27.905072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:27.917935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 508/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:27.924341 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:28.035355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238660b-17b2-4cc9-ba4e-4c794d3d80c6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:28.075330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:28.076081 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:28.076447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:28.076820 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:28.085308 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:28.086727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:28.098041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:28.098978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:28.100395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 530/1038] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:06:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:28.107322 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:28.210600 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feae8a-a39e-416d-9fb2-037852bcb8aa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:28.245369 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:28.245979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:28.246314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:28.246592 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:28.247334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:28.247647 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:28.249064 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:28.259805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:28.276398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 509/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:06:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:28.278407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:28.293120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:28.388179 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41657eb0-df87-48bc-b247-2b247b6c3fec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:28.421576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:28.422178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:28.422455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:28.422728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:28.492469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:28.493972 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:28.503917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:28.597093 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 531/1040] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:28.605180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:28.714902 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed674587-042f-45b1-b93a-3d7046c0cc6d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:28.751113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:28.751778 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:28.752059 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:28.752335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:28.762816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:28.764290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:28.779560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:28.780504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:28.781418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:28.781418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:28.781418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:28.783380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 510/1041] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:28.787220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:28.886237 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a01c-12fe-4013-8a85-13d4223e19b9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:28.897107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:28.923202 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:28.923844 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:28.924125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:28.924447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:28.925092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:28.925395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:28.927321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:28.938659 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 532/1042] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:28.944882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:28.996812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:29.056732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccbb75-0216-46e4-b01c-c62339eee6e8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:29.097761 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:29.098392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.098671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.098944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.105474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:29.111382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9ff29a8c-bbfd-4ad1-887f-10ccd8fdcf4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:29.125136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:29.126151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:29.127095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 511/1043] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:29.132816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:29.146886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390d2a-07cf-4036-ac21-389909accfe8 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.147172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390d2a-07cf-4036-ac21-389909accfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.147483 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15390d2a-07cf-4036-ac21-389909accfe8 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.148931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 512/1044] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:29.155035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.155994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.156476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.179587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:29.181646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:29.238925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859142-17c5-4a39-ad59-7218d1109386 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:29.255629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 513/1045] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:29.275029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:29.275661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.275942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.276216 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.276897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:29.277203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:29.278615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:29.295370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 533/1046] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:29.297353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:29.337953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:29.400338 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3c943-2222-4fb8-9d82-49b9bce888e5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:29.434336 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:29.434945 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.435225 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.435525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.436248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] RBAC: Adding query filter params (role.id=6ec1c3ef130347d4bbfdb77a7776246c, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=8d7573ee32f6443892a361f012c2f204) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:29.436557 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:29.437974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:29.451593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 514/1047] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:29 2022] GET /identity/v3/role_assignments?user.id=8d7573ee32f6443892a361f012c2f204&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6ec1c3ef130347d4bbfdb77a7776246c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:29.526788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:29.557620 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:29.637326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f2167-fca3-4ecc-8414-0a1e46b58ba2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:29.671366 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:29.672001 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.672279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.672550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.683317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:29.688111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:29.689043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:29.689922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:29.692028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 534/1048] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:29.695882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:29.796334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294db7c-6d93-4d5a-817a-7da6c74db811 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:29.797762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:29.832670 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:29.833330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.833612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.833887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.834575 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:29.834882 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:29.836342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2eae-6fec-4e59-ae80-db2def8f8ede None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:29.851587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 515/1049] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:29.853336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:29.951263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:29.954934 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492e17a-3554-49bb-909b-425b6b97fcec None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.955220 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492e17a-3554-49bb-909b-425b6b97fcec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.955520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492e17a-3554-49bb-909b-425b6b97fcec None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.957006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 516/1050] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:29.962613 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:29.962898 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:29.963169 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:29.966601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d2d80-5052-4be8-8865-2083a23ad51a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:29.980837 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:29.982197 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:30.001872 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.002499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.002772 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.003039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] PATH_INFO: `/v3/users/780fb42530d145259542228f9baa3378` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.018133 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] RBAC: Authorizing `identity:get_user(user_id=780fb42530d145259542228f9baa3378)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.019611 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.026958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 535/1051] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:29 2022] GET /identity/v3/users/780fb42530d145259542228f9baa3378 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:30.028330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.059391 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 517/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:30.136784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af644c-3aae-495f-9a18-aa1760a1bbf6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.170694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.171325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.171614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.171891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.187026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:30.239676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.241114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.325449 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.344085 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 536/1053] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 19:06:30.350898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.396922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:30.434968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:30.442328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30d31b-a624-4847-af26-850e6fc0ab66 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.454421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21647a3b-1113-44db-8226-c63d350512d0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.479005 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.479675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.479958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.480235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] PATH_INFO: `/v3/services/image` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.480820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.482188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:30.491217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:30.492224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:30.493061 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 518/1054] 10.222.0.18 () {56 vars in 1105 bytes} [Fri Sep 16 19:06:30 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:30.498057 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.498999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.499662 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.499942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.500214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.501405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] RBAC: Authorizing `identity:get_project(project_id=628ce272c8874186b0ae6e4805087345)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.502818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.509417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 537/1055] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:30 2022] GET /identity/v3/projects/628ce272c8874186b0ae6e4805087345 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:30.510213 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.607534 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e1626-37ae-4cec-9f53-4575b6f829fd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.616008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c27168-9c65-4edc-86a1-382f0fa26146 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.630819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:30.652308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.652930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.653205 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.653478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.653807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.654103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] RBAC: Adding query filter params (name=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:30.654406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.654514 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.654681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.654951 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.655701 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=628ce272c8874186b0ae6e4805087345, user.id=780fb42530d145259542228f9baa3378) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:30.655817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.656013 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.657410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.663830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 538/1056] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:30 2022] GET /identity/v3/role_assignments?user.id=780fb42530d145259542228f9baa3378&scope.project.id=628ce272c8874186b0ae6e4805087345&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:30.664492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 519/1057] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:30.670467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.782444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb903af-0fc0-4995-8637-d6fb602d6ca8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.819468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.820090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.820372 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.820647 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.821385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:30.821692 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.823128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89f44b-2bc6-4c9d-8a02-f21477510cc4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:30.828705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 539/1058] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:30.835122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:30.860566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:30.938207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752d899-46eb-42a2-a939-4f270b508d4c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:30.972130 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bda8-5cb1-4c90-90e4-5a9207df296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:30.972954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bda8-5cb1-4c90-90e4-5a9207df296d None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:30.973231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bda8-5cb1-4c90-90e4-5a9207df296d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:30.973498 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bda8-5cb1-4c90-90e4-5a9207df296d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:30.974159 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2bda8-5cb1-4c90-90e4-5a9207df296d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:30.975609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2bda8-5cb1-4c90-90e4-5a9207df296d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:31.003330 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 520/1059] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:06:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:31.620205 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ab26e-5d11-4d82-b036-e03f2a63ce14 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:31.620724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ab26e-5d11-4d82-b036-e03f2a63ce14 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:31.620819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ab26e-5d11-4d82-b036-e03f2a63ce14 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:31.622268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 540/1060] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:31.627093 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:31.627523 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:31.627682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:31.646028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:31.647203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:31.719770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 521/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:31.991166 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ab26e-5d11-4d82-b036-e03f2a63ce14 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:32.036279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:32.102590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ab26e-5d11-4d82-b036-e03f2a63ce14 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:32.136666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:32.137293 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.137574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.137848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.148364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:32.153198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:32.154141 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:32.154999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 541/1062] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Sep 16 19:06:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:32.160217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:32.163461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:32.261233 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916adb0d-eb8c-42e2-a8c8-63dbe1d24764 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:32.294771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:32.295411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.295691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.295962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.296620 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:32.296922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:32.298277 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:32.306468 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 522/1063] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:06:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:32.312468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:32.413394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41889243-1f11-4245-82f5-66cd9405b5cd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:32.414907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:32.448393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:32.449019 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.449299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.449575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.460145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:32.461582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:32.476245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:32.477180 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:32.478114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:32.478114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:32.478114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:32.478114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 542/1064] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:32.484124 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:32.551960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:32.588867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb074c-e979-41a7-8c11-09e3a7e2dc1c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:32.624100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:32.624714 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.624995 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.625271 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.625912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:32.626217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:32.627609 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:32.640038 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 523/1065] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:32.645769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:32.754446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4e43-4898-43ad-aa01-71a5f9ac4840 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:32.789107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:32.789735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.790079 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.790360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.798286 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:32.800070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-55a452d6-d173-4bef-840a-caa149985c49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:32.811804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:32.812792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:06:32.813835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:32.815376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 543/1066] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:06:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:32.818958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:32.836421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb36e5-63a5-45e1-a77c-ad173c071f88 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.836846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb36e5-63a5-45e1-a77c-ad173c071f88 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.837130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb36e5-63a5-45e1-a77c-ad173c071f88 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.838756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 544/1067] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:32.843752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.844035 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.844306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.868584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:32.869982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:32.880348 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:32.926492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93399461-ccf9-4965-a169-66560fe3f17b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:32.946715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 545/1068] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:32.954537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:32.964751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:32.965365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:32.965646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:32.965919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:32.966642 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:32.966946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:32.968415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:32.985896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 524/1069] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:06:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:32.992402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.099597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11017ee2-e055-4cbf-a759-244b51fc02b2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.134617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.135323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.135615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.135891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.145970 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.147495 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.150285 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-687610b5-96bc-4419-b434-c11e24debac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:33.163996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:33.164942 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:33.165870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:33.165870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:33.165870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:33.165870 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 546/1070] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:33.170824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.180244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e16373-b43f-42af-9bb1-d69d37864a9b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.180533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e16373-b43f-42af-9bb1-d69d37864a9b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.180810 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e16373-b43f-42af-9bb1-d69d37864a9b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.182127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 547/1071] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:06:33.187111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.187435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.187709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.207488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:33.208679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:33.271637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c467-09b1-4992-bb65-269484eddebb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.275217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 548/1072] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:33.276067 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.310860 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.311502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.311884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.312048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.312685 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:33.312991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.314349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.330521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 525/1073] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:33.331474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.378472 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.386887 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5c2d0-71af-44a9-b5ec-be7159fbaa85 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.414051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.424217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.424846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.425129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.425404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.435956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:33.436077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fdbf6-0329-4a71-b7b7-0244e52c16a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:33.440755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:33.441756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:33.442623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 549/1074] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:33.448526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.475643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.476262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.476537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.476820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.483354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.484775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:33.496050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:33.496956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:33.497891 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 526/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:33.503128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.555675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83af4e4-acba-436a-9860-81ff51644069 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.588437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.588964 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.589238 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.589505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.590151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:33.590445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.591822 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.600233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 550/1076] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:33.605549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.611876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a8683-b86c-4720-a347-3a70e341aa7e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.614419 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.645847 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.646474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.646726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.647003 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.647721 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:33.648030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.649451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.655670 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.661982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 527/1077] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:33.662780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.705948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91797bd1-e231-41f9-9a82-47c7b0e05b44 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.740344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.740944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.741221 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.741496 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.762411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ab230-af50-4527-94b2-c15895676a06 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.796360 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.796989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.797270 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.797542 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] PATH_INFO: `/v3/users/780fb42530d145259542228f9baa3378` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.798668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] RBAC: Authorizing `identity:get_user(user_id=780fb42530d145259542228f9baa3378)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.800127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.802341 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 528/1078] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/users/780fb42530d145259542228f9baa3378 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:33.803004 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.808050 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.809491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.831949 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.856332 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:33.899367 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fdd36-30e7-452c-884e-c3f9735084c1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:33.909184 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 551/1079] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:33.909867 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:33.935524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:33.936182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:33.936461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:33.936734 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/8d7573ee32f6443892a361f012c2f204/roles/6ec1c3ef130347d4bbfdb77a7776246c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:33.938888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=8d7573ee32f6443892a361f012c2f204, role_id=6ec1c3ef130347d4bbfdb77a7776246c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:33.940353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:33.958794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 529/1080] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:06:33 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/8d7573ee32f6443892a361f012c2f204/roles/6ec1c3ef130347d4bbfdb77a7776246c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:06:33.964290 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:34.027161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c63ad-a6b3-4182-aaa2-55982320b98b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:34.072789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:34.073509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:34.074155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:34.074344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:34.075823 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=628ce272c8874186b0ae6e4805087345)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:34.076307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:34.078028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:34.086445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 552/1081] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/projects/628ce272c8874186b0ae6e4805087345 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:34.087411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f739-7d23-4cbb-87a7-fbc36bbc857b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:34.087605 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:34.108225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:34.127411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:34.128070 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:34.128356 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:34.129147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:34.142817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:34.143552 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:34.161880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:34.162946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:34.163968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:34.163968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:34.163968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:34.163968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 530/1082] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:34.168926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:34.204279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e684ec9-3fcd-4057-bb1f-851b478f2dd2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:34.240548 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16237f82-9091-4989-b2dd-7bc9f3e201b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:34.241209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16237f82-9091-4989-b2dd-7bc9f3e201b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:34.241488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16237f82-9091-4989-b2dd-7bc9f3e201b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:34.241760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16237f82-9091-4989-b2dd-7bc9f3e201b9 None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345/users/780fb42530d145259542228f9baa3378/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:34.243728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16237f82-9091-4989-b2dd-7bc9f3e201b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=628ce272c8874186b0ae6e4805087345, user_id=780fb42530d145259542228f9baa3378, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:34.245152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16237f82-9091-4989-b2dd-7bc9f3e201b9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:34.267317 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 553/1083] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:06:34 2022] PUT /identity/v3/projects/628ce272c8874186b0ae6e4805087345/users/780fb42530d145259542228f9baa3378/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 19:06:34.281203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b28944-fc15-433b-942f-e54848561863 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:34.319196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:34.319852 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:34.320132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:34.320408 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:34.321071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:34.321374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:34.322781 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640efe0-edf0-4c34-bd29-6ace647821ac None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:34.337122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 531/1084] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:36.169817 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d3d7e-a48f-4df8-9dda-f5906ad475a4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.170388 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d3d7e-a48f-4df8-9dda-f5906ad475a4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.170388 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d3d7e-a48f-4df8-9dda-f5906ad475a4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.171707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 554/1085] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:36.177525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cb546-3f65-4df8-8d83-20eb1fb92517 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.177936 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cb546-3f65-4df8-8d83-20eb1fb92517 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.178070 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cb546-3f65-4df8-8d83-20eb1fb92517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.186008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:36.196400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-631cb546-3f65-4df8-8d83-20eb1fb92517 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:36.197569 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-631cb546-3f65-4df8-8d83-20eb1fb92517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:36.261871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 532/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:36.500164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf859-febc-4ea2-a872-8bed831fffad None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.500447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf859-febc-4ea2-a872-8bed831fffad None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.500719 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf859-febc-4ea2-a872-8bed831fffad None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.502040 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 555/1087] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:36.507180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.507502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.507780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.525731 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:36.526895 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:36.531789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf859-febc-4ea2-a872-8bed831fffad None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:36.593684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 533/1088] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:36.609084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:36.634346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf859-febc-4ea2-a872-8bed831fffad None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:36.666561 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:36.667178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.667480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.667751 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.677773 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:36.682504 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:36.683718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:36.684784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 556/1089] 10.222.0.18 () {56 vars in 1120 bytes} [Fri Sep 16 19:06:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:36.689484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:36.750508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:36.787054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf8e4b-da5c-4d82-b28a-2bafc3d0a665 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:36.819526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:36.820121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.820394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.820662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.821309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:36.821605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:36.822946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:36.830728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 534/1090] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:06:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:36.836808 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:36.866576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:36.936496 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9e85f-ffed-4dab-9e2d-955268ba0235 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:36.963755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a5473-152c-421b-8a8b-71adaf997b98 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:36.969798 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:36.970414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.970687 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:36.970956 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:36.974305 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:36.983010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:36.984502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:36.989480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:36.998509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:36.999138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:36.999376 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:37.000442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:37.001377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:37.001377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:37.001377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:37.001561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.001561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.001694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 557/1091] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:37.006339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.010752 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:37.015570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:37.016466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:37.017314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 535/1092] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:37.022589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.105206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3a886-7c37-42c4-b0fe-7f3ab5826206 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.121997 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577ded6-ff0e-4982-9bbf-48de3c4c8d92 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.139301 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.139935 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.140218 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.140492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.141128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:37.141527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.143797 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.155908 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.156515 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.156794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.157065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.161821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 558/1093] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:37.161971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:37.161971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.161971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.164220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.168257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 536/1094] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:37.174055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.212666 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.221154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.263512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f94c81-342d-48fd-8d7c-8c93e34e358e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.272245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ce1e-9b73-4736-83cf-f3d31b2f40a5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.299060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.299705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.299988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.300256 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.305581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.306202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.306631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.306901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] PATH_INFO: `/v3/users/780fb42530d145259542228f9baa3378` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.307247 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.308107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] RBAC: Authorizing `identity:get_user(user_id=780fb42530d145259542228f9baa3378)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.308717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.309511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.311079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 537/1095] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/users/780fb42530d145259542228f9baa3378 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:37.316770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:06:37.321444 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:37.322576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:37.323733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 559/1096] 10.222.0.18 () {56 vars in 1093 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:37.329171 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.418537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d015f-c0e3-48c6-93b6-1ee00e9d8168 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.431729 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1d23d-4403-44c7-97f5-fcf90d923604 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.435344 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.440773 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.453876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.454484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.454767 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.455040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.466289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.466886 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.467169 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.467474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.468196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:37.468500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.469920 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.487787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 560/1097] 10.222.0.18 () {56 vars in 1133 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:37.493742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.522093 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.523492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.590044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f499ba8-965f-49e2-b252-4bb25d1851bb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.618807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 538/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 19:06:37.622862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82017283-feee-413e-b930-f54d546d598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.623515 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82017283-feee-413e-b930-f54d546d598a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.623786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82017283-feee-413e-b930-f54d546d598a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.624058 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82017283-feee-413e-b930-f54d546d598a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.624871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.641414 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.658144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.693372 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82017283-feee-413e-b930-f54d546d598a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.694811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82017283-feee-413e-b930-f54d546d598a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.727863 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7c764-58bd-4e11-ac8f-5d5f61c0fce9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.772210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.773075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.773633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.774086 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.776144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] RBAC: Authorizing `identity:get_project(project_id=628ce272c8874186b0ae6e4805087345)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.777722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.780248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 539/1099] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/projects/628ce272c8874186b0ae6e4805087345 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:37.785813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.806641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 561/1100] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 19:06:37.812357 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82017283-feee-413e-b930-f54d546d598a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:37.879447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.884767 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:37.896872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f031-b9b9-4e5f-b9d3-8d20d74951a9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.925347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82017283-feee-413e-b930-f54d546d598a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:37.935754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.936436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.936778 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.937114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.937977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=628ce272c8874186b0ae6e4805087345, user.id=780fb42530d145259542228f9baa3378) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:37.938350 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.939924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:37.947625 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 540/1101] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/role_assignments?user.id=780fb42530d145259542228f9baa3378&scope.project.id=628ce272c8874186b0ae6e4805087345&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:37.969098 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:37.969756 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:37.970023 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:37.970291 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:37.982480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:37.983985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:06:38.000242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:38.006374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:06:38.007473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:06:38.007473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:06:38.007473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:38.007473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 562/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:06:38.007763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:38.128349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39da491-62a8-41cd-b69e-a8a325914f81 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:38.162203 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:38.162856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:38.163141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:38.163450 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:38.164198 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:38.164541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:38.166072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:38.178664 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 541/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:38.184755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:38.282648 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71e6b5-9719-4dd1-add1-005f4f53423d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:38.299350 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:38.316996 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:38.317623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:38.317904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:38.318178 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:38.324867 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:38.326241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:06:38.337296 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:38.338236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:06:38.339509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 563/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:38.344265 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:38.355756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:38.448911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba9dba-84f1-4192-9f10-65f486627b1b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:38.482794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:38.483429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:38.483713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:38.483987 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:38.484675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:38.484982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:38.486398 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b455b8-8244-4cfc-b986-5f4ee2a8c5b9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:38.497907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 542/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:38.504498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:38.603215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d35378-c1dd-4bcc-b4e6-573fb04f5ced None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:38.636228 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:38.636827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:38.637107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:38.637379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:38.638107 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] RBAC: Adding query filter params (role.id=6ec1c3ef130347d4bbfdb77a7776246c, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=8d7573ee32f6443892a361f012c2f204) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:38.638417 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:38.639871 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174254b-3c44-49c7-aa58-74f0ed53f646 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:38.646101 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 564/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:38 2022] GET /identity/v3/role_assignments?user.id=8d7573ee32f6443892a361f012c2f204&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6ec1c3ef130347d4bbfdb77a7776246c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:06:40.143148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf61ef-fe1f-492d-b173-b353737bb8dc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:40.143756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf61ef-fe1f-492d-b173-b353737bb8dc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:40.143756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf61ef-fe1f-492d-b173-b353737bb8dc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:40.145114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 543/1107] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:40.150175 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:40.150586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:40.150725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:40.168922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:40.170095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:40.203576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:40.233926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 565/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:40.505485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf61ef-fe1f-492d-b173-b353737bb8dc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:40.570253 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:40.603423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf61ef-fe1f-492d-b173-b353737bb8dc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:40.635820 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0656527-4255-487b-97eb-bdba85b02d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:40.636440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0656527-4255-487b-97eb-bdba85b02d6a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:40.636718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0656527-4255-487b-97eb-bdba85b02d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:40.636990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0656527-4255-487b-97eb-bdba85b02d6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:40.638086 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0656527-4255-487b-97eb-bdba85b02d6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:40.639532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0656527-4255-487b-97eb-bdba85b02d6a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:40.641079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 544/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:40.646800 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:40.742349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1a54-3d57-4eca-a1a5-ef7c7a03b87c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:40.774117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef386b-5351-457c-bf80-0738a0911c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:40.774647 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef386b-5351-457c-bf80-0738a0911c2a None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:40.774925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef386b-5351-457c-bf80-0738a0911c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:40.775194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef386b-5351-457c-bf80-0738a0911c2a None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:40.775863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef386b-5351-457c-bf80-0738a0911c2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:40.777263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef386b-5351-457c-bf80-0738a0911c2a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:40.797913 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 566/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:06:40 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:42.956922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fc8e-3139-4a37-8bb1-279a7b345d74 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:42.957504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fc8e-3139-4a37-8bb1-279a7b345d74 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:42.957504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fc8e-3139-4a37-8bb1-279a7b345d74 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:42.958816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 545/1111] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:42.964019 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:42.964473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:42.964572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:42.982677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:42.983874 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:43.006233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:43.048433 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 567/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:43.318132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019fc8e-3139-4a37-8bb1-279a7b345d74 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:43.434267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019fc8e-3139-4a37-8bb1-279a7b345d74 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:43.469093 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:43.469710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:43.469985 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:43.470257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] PATH_INFO: `/v3/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:43.471643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] RBAC: Authorizing `identity:get_role(role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:43.473537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:43.474826 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 546/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:43 2022] GET /identity/v3/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:43.480806 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:43.575358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9d82-b433-4255-a4c5-fd6c6ad551be None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:43.608018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:43.608627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:43.608904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:43.609177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] PATH_INFO: `/v3/users/227ab9aa8bc5440592126d32cf41dea5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:43.622021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] RBAC: Authorizing `identity:get_user(user_id=227ab9aa8bc5440592126d32cf41dea5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:43.623473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:43.625018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 568/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:43 2022] GET /identity/v3/users/227ab9aa8bc5440592126d32cf41dea5 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:43.631053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:43.634704 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:43.733766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09234a01-1686-481e-a73d-2ef2e3ff7cf2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:43.771207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:43.771832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:43.772108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:43.772378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:43.836019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:43.837383 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:43.839836 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:43.930726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 547/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:43.936626 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:43.953814 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:44.033024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105ff70-21f8-4a89-848f-fab29237050f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:44.070231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c633d9-8625-4a8b-ae63-a274fdc3a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:44.070860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c633d9-8625-4a8b-ae63-a274fdc3a35f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:44.071137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c633d9-8625-4a8b-ae63-a274fdc3a35f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:44.071452 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c633d9-8625-4a8b-ae63-a274fdc3a35f None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:44.072563 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c633d9-8625-4a8b-ae63-a274fdc3a35f None admin] RBAC: Authorizing `identity:get_project(project_id=628ce272c8874186b0ae6e4805087345)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:44.073978 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c633d9-8625-4a8b-ae63-a274fdc3a35f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:44.075625 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 569/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:43 2022] GET /identity/v3/projects/628ce272c8874186b0ae6e4805087345 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:44.082326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:44.177506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea76c7a-ca63-4d63-a8d3-d6c844d0d38f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:44.179673 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:44.210318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:44.210911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:44.211190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:44.211500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:44.212211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] RBAC: Adding query filter params (role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=628ce272c8874186b0ae6e4805087345, user.id=227ab9aa8bc5440592126d32cf41dea5) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:44.212516 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:44.213924 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c29b4-2db8-4731-b57d-d01908d82e22 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:44.219033 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 548/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:44 2022] GET /identity/v3/role_assignments?user.id=227ab9aa8bc5440592126d32cf41dea5&scope.project.id=628ce272c8874186b0ae6e4805087345&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:46.304491 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86c667-637d-44bd-bade-b7fe3f2fc29f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:46.305037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86c667-637d-44bd-bade-b7fe3f2fc29f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:46.305037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86c667-637d-44bd-bade-b7fe3f2fc29f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:46.306358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 570/1118] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:46.311428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542833e7-326b-4847-ba86-68a852454c54 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:46.311801 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542833e7-326b-4847-ba86-68a852454c54 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:46.311979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542833e7-326b-4847-ba86-68a852454c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:46.330213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-542833e7-326b-4847-ba86-68a852454c54 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:46.331329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-542833e7-326b-4847-ba86-68a852454c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:46.395264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 549/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:46.661909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86c667-637d-44bd-bade-b7fe3f2fc29f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:46.744489 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:46.760775 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86c667-637d-44bd-bade-b7fe3f2fc29f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:46.793469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:46.794090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:46.794370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:46.794647 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] PATH_INFO: `/v3/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:46.796056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] RBAC: Authorizing `identity:get_role(role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:46.797406 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:46.798689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 571/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:46 2022] GET /identity/v3/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:46.805099 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542833e7-326b-4847-ba86-68a852454c54 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:46.895674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:46.899591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542833e7-326b-4847-ba86-68a852454c54 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:46.932645 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:46.933204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:46.933481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:46.933751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] PATH_INFO: `/v3/users/227ab9aa8bc5440592126d32cf41dea5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:46.934844 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=227ab9aa8bc5440592126d32cf41dea5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:46.936285 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:46.937816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 550/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:46 2022] GET /identity/v3/users/227ab9aa8bc5440592126d32cf41dea5 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:46.943395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:47.039473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0deab13-84aa-4128-b12b-a5ad1fc72701 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:47.071533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3dd37-be24-4b77-a647-6a8edeba377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:47.072154 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3dd37-be24-4b77-a647-6a8edeba377b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:47.072434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3dd37-be24-4b77-a647-6a8edeba377b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:47.072709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3dd37-be24-4b77-a647-6a8edeba377b None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:47.074031 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3dd37-be24-4b77-a647-6a8edeba377b None admin] RBAC: Authorizing `identity:get_project(project_id=628ce272c8874186b0ae6e4805087345)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:47.075234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3dd37-be24-4b77-a647-6a8edeba377b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:47.076854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 572/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:46 2022] GET /identity/v3/projects/628ce272c8874186b0ae6e4805087345 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:47.083232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:47.181426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2dd24-b2ab-4bc9-9023-e4f51c56a5f9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:47.213741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab987b1-86f0-417a-8d72-e2ce63591f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:47.214329 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab987b1-86f0-417a-8d72-e2ce63591f27 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:47.214602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab987b1-86f0-417a-8d72-e2ce63591f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:47.214868 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab987b1-86f0-417a-8d72-e2ce63591f27 None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345/users/227ab9aa8bc5440592126d32cf41dea5/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:47.216960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab987b1-86f0-417a-8d72-e2ce63591f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=628ce272c8874186b0ae6e4805087345, user_id=227ab9aa8bc5440592126d32cf41dea5, role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:47.218371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab987b1-86f0-417a-8d72-e2ce63591f27 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:47.231313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 551/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:06:47 2022] PUT /identity/v3/projects/628ce272c8874186b0ae6e4805087345/users/227ab9aa8bc5440592126d32cf41dea5/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:06:49.331001 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ce9d3-fb55-49d4-a75e-12788fe1905a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:49.332053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ce9d3-fb55-49d4-a75e-12788fe1905a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:49.332053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ce9d3-fb55-49d4-a75e-12788fe1905a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:49.333337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 573/1124] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:49.338441 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:49.338874 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:49.338974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:49.357243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:49.358403 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:49.377952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:49.423540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 552/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:49.697937 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ce9d3-fb55-49d4-a75e-12788fe1905a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:49.719158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:49.807456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ce9d3-fb55-49d4-a75e-12788fe1905a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:49.840177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:49.840797 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:49.841073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:49.841346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] PATH_INFO: `/v3/roles/211de5da0ddf45a9aa3fde1265c291ff` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:49.842722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] RBAC: Authorizing `identity:get_role(role_id=211de5da0ddf45a9aa3fde1265c291ff)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:49.844111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:49.845407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 574/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:49 2022] GET /identity/v3/roles/211de5da0ddf45a9aa3fde1265c291ff => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:49.851790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:49.946642 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf3108-14f9-433e-95d1-fb2db075e165 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:49.979671 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:49.980232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:49.980512 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:49.980786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] PATH_INFO: `/v3/users/227ab9aa8bc5440592126d32cf41dea5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:49.981886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] RBAC: Authorizing `identity:get_user(user_id=227ab9aa8bc5440592126d32cf41dea5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:49.983331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:49.984865 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 553/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:49 2022] GET /identity/v3/users/227ab9aa8bc5440592126d32cf41dea5 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:49.990713 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:50.074194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:50.086661 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98afc2-c45a-4394-834e-e98383bbf003 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:50.122473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:50.123065 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:50.123375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:50.123651 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:50.193910 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:50.195370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:50.277132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:50.290278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 575/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:50.296190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:50.320756 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:50.392506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa05ec1-adeb-488b-aa4c-a15e182680db None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:50.425172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cf61a-1434-431e-b222-18f3bfe720e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:50.425797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cf61a-1434-431e-b222-18f3bfe720e3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:50.426077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cf61a-1434-431e-b222-18f3bfe720e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:50.426347 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cf61a-1434-431e-b222-18f3bfe720e3 None admin] PATH_INFO: `/v3/projects/628ce272c8874186b0ae6e4805087345` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:50.427485 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cf61a-1434-431e-b222-18f3bfe720e3 None admin] RBAC: Authorizing `identity:get_project(project_id=628ce272c8874186b0ae6e4805087345)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:50.428906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cf61a-1434-431e-b222-18f3bfe720e3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:50.430513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 554/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:50 2022] GET /identity/v3/projects/628ce272c8874186b0ae6e4805087345 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:50.437883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:50.534488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c83653-e7e6-4741-94bf-839395f4f8a1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:50.567187 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:50.567799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:50.568079 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:50.568354 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:50.569071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] RBAC: Adding query filter params (role.id=211de5da0ddf45a9aa3fde1265c291ff, scope.project.id=628ce272c8874186b0ae6e4805087345, user.id=227ab9aa8bc5440592126d32cf41dea5) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:50.569380 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:50.570796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929850e7-7602-45bc-89eb-2e9e04d06af4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:50.576419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 576/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:50 2022] GET /identity/v3/role_assignments?user.id=227ab9aa8bc5440592126d32cf41dea5&scope.project.id=628ce272c8874186b0ae6e4805087345&role.id=211de5da0ddf45a9aa3fde1265c291ff => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:52.706285 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b05d-8f4d-4145-a924-0b27f3713a77 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:52.706852 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b05d-8f4d-4145-a924-0b27f3713a77 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:52.706944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b05d-8f4d-4145-a924-0b27f3713a77 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:52.708430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 555/1131] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:52.713697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:52.714123 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:52.714244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:52.732360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:52.733523 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:52.789676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:52.814111 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 577/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:53.087462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b05d-8f4d-4145-a924-0b27f3713a77 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:53.186941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:53.197313 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b05d-8f4d-4145-a924-0b27f3713a77 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:53.229665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb572aa4-8ac5-4e0f-bee0-b25853a4788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:53.230291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb572aa4-8ac5-4e0f-bee0-b25853a4788f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:53.230570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb572aa4-8ac5-4e0f-bee0-b25853a4788f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:53.230842 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb572aa4-8ac5-4e0f-bee0-b25853a4788f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:53.231977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb572aa4-8ac5-4e0f-bee0-b25853a4788f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:53.233399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb572aa4-8ac5-4e0f-bee0-b25853a4788f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:53.234960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 556/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:53.241616 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:53.334525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c4a41-6291-4817-a7b0-e2ef0956ff70 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:53.366670 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b416b7-0525-4a3e-bee6-0ec6d1de05f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:53.367250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b416b7-0525-4a3e-bee6-0ec6d1de05f0 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:53.367552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b416b7-0525-4a3e-bee6-0ec6d1de05f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:53.367821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b416b7-0525-4a3e-bee6-0ec6d1de05f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:53.368456 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b416b7-0525-4a3e-bee6-0ec6d1de05f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:53.369848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b416b7-0525-4a3e-bee6-0ec6d1de05f0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:53.397863 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 578/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:06:53 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:55.574922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe853f76-d400-4768-ae76-db0898e4dc96 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:55.575469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe853f76-d400-4768-ae76-db0898e4dc96 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:55.575469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe853f76-d400-4768-ae76-db0898e4dc96 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:55.576839 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 557/1135] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:55.582143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:55.582565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:55.582686 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:55.600613 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:55.601777 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:55.627518 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:55.666698 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 579/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:06:55.939323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe853f76-d400-4768-ae76-db0898e4dc96 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:56.036532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe853f76-d400-4768-ae76-db0898e4dc96 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:56.068774 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37b771-3dc7-4bb3-a525-ef7f333363a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:56.069405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b771-3dc7-4bb3-a525-ef7f333363a4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:56.069684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b771-3dc7-4bb3-a525-ef7f333363a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:56.069958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37b771-3dc7-4bb3-a525-ef7f333363a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:56.071063 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37b771-3dc7-4bb3-a525-ef7f333363a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:56.072541 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37b771-3dc7-4bb3-a525-ef7f333363a4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:56.074088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 558/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:56.079858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:56.174977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec0c26-0332-4f5e-ac9c-c89025e9b732 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:56.206936 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cb018-6cf0-426e-b8f4-7ce14c420add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:56.207493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cb018-6cf0-426e-b8f4-7ce14c420add None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:56.207770 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cb018-6cf0-426e-b8f4-7ce14c420add None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:56.208037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cb018-6cf0-426e-b8f4-7ce14c420add None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:56.208678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cb018-6cf0-426e-b8f4-7ce14c420add None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:56.210077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cb018-6cf0-426e-b8f4-7ce14c420add None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:56.230070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 580/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:06:56 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:06:58.377655 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e13768-4576-45fd-9737-f0861927ccef None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:58.378173 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e13768-4576-45fd-9737-f0861927ccef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:58.378173 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e13768-4576-45fd-9737-f0861927ccef None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:58.379500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 559/1139] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:06:58.384737 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:58.385147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:58.385274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:58.403500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:06:58.404658 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:06:58.418685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:58.469568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 581/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:06:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:06:58.741132 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e13768-4576-45fd-9737-f0861927ccef None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:58.773862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:58.850871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e13768-4576-45fd-9737-f0861927ccef None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:58.883886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:58.884511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:58.884794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:58.885069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:58.895504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:06:58.900305 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:06:58.901451 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:06:58.902561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 560/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:06:58.907451 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:59.002114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed470ee6-23a9-457e-9805-f5a31f4e6577 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:59.034731 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:59.035335 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:59.035622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:59.035893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:59.036545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:59.036846 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:59.038204 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:59.045858 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 582/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:59.052167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:59.146659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:59.160320 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd99943-936a-4656-8cc4-91349623f4f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:59.194392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:59.195015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:59.195306 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:59.195591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] PATH_INFO: `/v3/users/dcabeae6a6a44651bd736129751e6aa4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:59.208840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] RBAC: Authorizing `identity:get_user(user_id=dcabeae6a6a44651bd736129751e6aa4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:59.210266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:59.211899 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 561/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:06:59 2022] GET /identity/v3/users/dcabeae6a6a44651bd736129751e6aa4 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:59.217776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:59.233331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:59.315198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413884d-2bf1-4903-aade-29be51dd0fe0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:59.347575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73870e31-3332-4a10-b005-449364041514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:59.348157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73870e31-3332-4a10-b005-449364041514 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:59.348435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73870e31-3332-4a10-b005-449364041514 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:59.348707 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73870e31-3332-4a10-b005-449364041514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:59.412361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73870e31-3332-4a10-b005-449364041514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:59.413783 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73870e31-3332-4a10-b005-449364041514 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:59.426521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:59.509939 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 583/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:06:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:06:59.515888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:59.611207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a863c2-9590-4a8c-b4c6-1f0bd0149986 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:59.643656 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff641-3aee-4c1a-8523-047d7b78362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:59.644831 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff641-3aee-4c1a-8523-047d7b78362d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:59.645111 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff641-3aee-4c1a-8523-047d7b78362d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:59.645386 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff641-3aee-4c1a-8523-047d7b78362d None admin] PATH_INFO: `/v3/projects/cc9594a97a964c4085390784dd36a4e9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:59.646492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff641-3aee-4c1a-8523-047d7b78362d None admin] RBAC: Authorizing `identity:get_project(project_id=cc9594a97a964c4085390784dd36a4e9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:59.647941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff641-3aee-4c1a-8523-047d7b78362d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:59.649562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 562/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:06:59 2022] GET /identity/v3/projects/cc9594a97a964c4085390784dd36a4e9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:06:59.655606 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73870e31-3332-4a10-b005-449364041514 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:06:59.750845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73870e31-3332-4a10-b005-449364041514 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:06:59.763269 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:06:59.784528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:06:59.785122 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:06:59.785399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:06:59.785672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:06:59.786387 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=cc9594a97a964c4085390784dd36a4e9, user.id=dcabeae6a6a44651bd736129751e6aa4) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:06:59.786693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:06:59.788131 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bc414-f984-4fb9-b227-4b06195fedf9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:06:59.793263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 584/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:06:59 2022] GET /identity/v3/role_assignments?user.id=dcabeae6a6a44651bd736129751e6aa4&scope.project.id=cc9594a97a964c4085390784dd36a4e9&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:01.911749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef719e7-116a-4722-be2c-3b7fdbfc4a4d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:01.912295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef719e7-116a-4722-be2c-3b7fdbfc4a4d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:01.912295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef719e7-116a-4722-be2c-3b7fdbfc4a4d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:01.913604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 563/1147] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:01.918790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:01.919222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:01.919364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:01.937583 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:01.938776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:02.004049 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 585/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:02.275268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef719e7-116a-4722-be2c-3b7fdbfc4a4d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:02.290601 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:02.373199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef719e7-116a-4722-be2c-3b7fdbfc4a4d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:02.405730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:02.406340 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:02.406615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:02.406883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:02.416397 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:07:02.421122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:07:02.422214 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:07:02.423245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 564/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:07:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:07:02.428206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:02.522745 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c39d34-6e56-4319-905b-e4024f941d02 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:02.529663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:02.555920 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:02.556512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:02.556786 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:02.557056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:02.557700 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:02.557995 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:02.559350 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:02.567468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 586/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:02.573263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:02.646176 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:02.671240 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488c5b-f5a7-44d0-aead-49b36ad48c13 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:02.703653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:02.704260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:02.704536 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:02.704805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] PATH_INFO: `/v3/users/dcabeae6a6a44651bd736129751e6aa4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:02.705890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] RBAC: Authorizing `identity:get_user(user_id=dcabeae6a6a44651bd736129751e6aa4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:02.707829 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:02.709492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 565/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:07:02 2022] GET /identity/v3/users/dcabeae6a6a44651bd736129751e6aa4 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:02.715472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:02.811242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d50968-2b8f-41b2-b1d6-c84cd4a7fc4a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:02.843125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d94f9-9bd1-4ced-9b1f-c08d00d5905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:02.843764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d94f9-9bd1-4ced-9b1f-c08d00d5905d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:02.844045 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d94f9-9bd1-4ced-9b1f-c08d00d5905d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:02.844324 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d94f9-9bd1-4ced-9b1f-c08d00d5905d None admin] PATH_INFO: `/v3/projects/cc9594a97a964c4085390784dd36a4e9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:02.845423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d94f9-9bd1-4ced-9b1f-c08d00d5905d None admin] RBAC: Authorizing `identity:get_project(project_id=cc9594a97a964c4085390784dd36a4e9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:02.846852 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d94f9-9bd1-4ced-9b1f-c08d00d5905d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:02.848463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 587/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:07:02 2022] GET /identity/v3/projects/cc9594a97a964c4085390784dd36a4e9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:02.854149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:02.949754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0585b-3a89-455b-8c64-513348ecb6ab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:02.981963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c993d4-7458-47c6-a46d-a464eab6b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:02.982604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c993d4-7458-47c6-a46d-a464eab6b78f None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:02.982883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c993d4-7458-47c6-a46d-a464eab6b78f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:02.983157 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c993d4-7458-47c6-a46d-a464eab6b78f None admin] PATH_INFO: `/v3/projects/cc9594a97a964c4085390784dd36a4e9/users/dcabeae6a6a44651bd736129751e6aa4/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:02.985060 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c993d4-7458-47c6-a46d-a464eab6b78f None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9594a97a964c4085390784dd36a4e9, user_id=dcabeae6a6a44651bd736129751e6aa4, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:02.986474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c993d4-7458-47c6-a46d-a464eab6b78f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:03.000166 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 566/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:07:02 2022] PUT /identity/v3/projects/cc9594a97a964c4085390784dd36a4e9/users/dcabeae6a6a44651bd736129751e6aa4/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:07:05.116575 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5899-a845-430a-9eb8-da5559aae0e5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:05.117076 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5899-a845-430a-9eb8-da5559aae0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:05.117173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5899-a845-430a-9eb8-da5559aae0e5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:05.118442 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 588/1154] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:05.123573 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:05.124007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:05.124106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:05.131016 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:05.143133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:05.145105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:05.222436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 567/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:05.504189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5899-a845-430a-9eb8-da5559aae0e5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:05.540690 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:05.604410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5899-a845-430a-9eb8-da5559aae0e5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:05.636545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:05.637158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:05.637435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:05.637704 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:05.648363 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:07:05.653121 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:07:05.654251 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:07:05.655672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 589/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:07:05.660781 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:05.755589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ff26-36da-460c-81fd-8a2b9006c503 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:05.783384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:05.789015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:05.789614 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:05.789892 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:05.790164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:05.790816 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:05.791143 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:05.792526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:05.800317 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 568/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:05.806091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:05.899966 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:05.905578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9398557-89f7-4c96-a4b1-edad737a5f6f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:05.937310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:05.937923 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:05.938198 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:05.938475 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] PATH_INFO: `/v3/users/dcabeae6a6a44651bd736129751e6aa4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:05.939601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] RBAC: Authorizing `identity:get_user(user_id=dcabeae6a6a44651bd736129751e6aa4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:05.941580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:05.943108 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 590/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:07:05 2022] GET /identity/v3/users/dcabeae6a6a44651bd736129751e6aa4 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:05.948889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:06.046410 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec313c-d4b2-4d95-ac2f-1843c4031e94 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:06.079239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:06.079871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:06.080155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:06.080431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:06.138476 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:06.146207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:06.147683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:06.242098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 569/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:07:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:07:06.248144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:06.344422 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e5c73-c730-48c3-944f-7d53a69940a9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:06.376674 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfb3c3-64ae-410d-a070-88eb366d2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:06.377288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfb3c3-64ae-410d-a070-88eb366d2ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:06.377570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfb3c3-64ae-410d-a070-88eb366d2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:06.377845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfb3c3-64ae-410d-a070-88eb366d2ba1 None admin] PATH_INFO: `/v3/projects/cc9594a97a964c4085390784dd36a4e9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:06.379507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfb3c3-64ae-410d-a070-88eb366d2ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9594a97a964c4085390784dd36a4e9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:06.380925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfb3c3-64ae-410d-a070-88eb366d2ba1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:06.382508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 591/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:07:06 2022] GET /identity/v3/projects/cc9594a97a964c4085390784dd36a4e9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:06.388546 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:06.476621 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:06.483578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b814f5b-d22f-471d-99c5-1364a5a48fa2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:06.515766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:06.516357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:06.516638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:06.516910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:06.517622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=cc9594a97a964c4085390784dd36a4e9, user.id=dcabeae6a6a44651bd736129751e6aa4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:06.517935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:06.520022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcaf9e-d206-4e94-9b18-76a2e7c25065 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:06.525549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 570/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:07:06 2022] GET /identity/v3/role_assignments?user.id=dcabeae6a6a44651bd736129751e6aa4&scope.project.id=cc9594a97a964c4085390784dd36a4e9&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:08.699028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c449447-9ac5-48a1-a631-754014ebd891 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:08.699619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c449447-9ac5-48a1-a631-754014ebd891 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:08.699619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c449447-9ac5-48a1-a631-754014ebd891 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:08.700918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 592/1162] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:08.706587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:08.707027 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:08.707131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:08.725238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:08.726397 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:08.789190 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 571/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:09.061164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c449447-9ac5-48a1-a631-754014ebd891 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:09.092512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:09.165676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c449447-9ac5-48a1-a631-754014ebd891 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:09.197326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:09.197916 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:09.198189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:09.198455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:09.261918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:09.263383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:09.292367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:09.358055 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 593/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:07:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:07:09.364465 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:09.457182 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:09.470616 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b8cf2-5813-4c38-8f60-0abd3ea9ef5e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:09.507871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:09.508504 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:09.508805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:09.509065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:09.515527 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:09.516959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d8d800c2-04f1-486a-8859-851d1a4c2e05 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:07:09.528561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:07:09.529758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:07:09.530810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 572/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:07:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:07:09.536617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:09.631021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83187a08-dde0-46a4-8bfa-99b1723de08d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:09.659214 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:09.664375 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:09.664993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:09.665267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:09.665533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:09.666159 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:09.666455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:09.667855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19640-a1af-4b25-bd07-b45d9e5ffa29 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:09.675430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 594/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Fri Sep 16 19:07:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:11.802565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595fc15-a042-43ed-976c-7d064a730903 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:11.803113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595fc15-a042-43ed-976c-7d064a730903 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:11.803113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595fc15-a042-43ed-976c-7d064a730903 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:11.804457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 573/1167] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:11.809594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7c9c-8ee9-4617-83d5-d1ae0395db88 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:11.809973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7c9c-8ee9-4617-83d5-d1ae0395db88 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:11.810151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7c9c-8ee9-4617-83d5-d1ae0395db88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:11.828435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f87f7c9c-8ee9-4617-83d5-d1ae0395db88 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:11.829615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f87f7c9c-8ee9-4617-83d5-d1ae0395db88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:11.892915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 595/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:12.158751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595fc15-a042-43ed-976c-7d064a730903 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:12.258848 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595fc15-a042-43ed-976c-7d064a730903 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:12.290620 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9585374-e003-4792-adc6-a9d976cf8028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:12.291208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9585374-e003-4792-adc6-a9d976cf8028 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:12.291618 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9585374-e003-4792-adc6-a9d976cf8028 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:12.291894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9585374-e003-4792-adc6-a9d976cf8028 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:12.292548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9585374-e003-4792-adc6-a9d976cf8028 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:12.293904 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9585374-e003-4792-adc6-a9d976cf8028 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:12.309588 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 574/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Fri Sep 16 19:07:12 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:07:14.446414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fe3c5-20bf-496c-bbe3-cd9d99e15264 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:14.446913 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fe3c5-20bf-496c-bbe3-cd9d99e15264 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:14.446913 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fe3c5-20bf-496c-bbe3-cd9d99e15264 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:14.448241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 596/1170] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:14.453367 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:14.453780 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:14.453920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:14.472220 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:14.473401 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:14.475328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:14.540687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 575/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:14.808926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fe3c5-20bf-496c-bbe3-cd9d99e15264 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:14.915222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fe3c5-20bf-496c-bbe3-cd9d99e15264 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:14.939574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:14.948207 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68d5c1-1269-413e-87b5-5e1443f0c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:14.948831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d5c1-1269-413e-87b5-5e1443f0c515 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:14.949112 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d5c1-1269-413e-87b5-5e1443f0c515 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:14.949389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68d5c1-1269-413e-87b5-5e1443f0c515 None admin] PATH_INFO: `/v3/domains/d99e90aa994748b298aa43195eeaf26d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:14.960339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68d5c1-1269-413e-87b5-5e1443f0c515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d99e90aa994748b298aa43195eeaf26d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:14.961765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68d5c1-1269-413e-87b5-5e1443f0c515 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:14.963364 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 597/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Fri Sep 16 19:07:14 2022] GET /identity/v3/domains/d99e90aa994748b298aa43195eeaf26d => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:14.969119 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:15.064457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923eec0-bb02-4392-8550-2adc532d1fee None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:15.098550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae296-4cfd-46ff-95ee-2c706882a757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:15.099144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae296-4cfd-46ff-95ee-2c706882a757 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:15.099454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae296-4cfd-46ff-95ee-2c706882a757 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:15.099727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae296-4cfd-46ff-95ee-2c706882a757 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:15.100381 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae296-4cfd-46ff-95ee-2c706882a757 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:15.101817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae296-4cfd-46ff-95ee-2c706882a757 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:15.110775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:15.132619 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 576/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:07:14 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:07:17.284679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270723c4-1c0e-4828-9317-e59ec7b1e07f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:17.285231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270723c4-1c0e-4828-9317-e59ec7b1e07f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:17.285620 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270723c4-1c0e-4828-9317-e59ec7b1e07f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:17.287902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 598/1174] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:17.294395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:17.294827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:17.294943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:17.313500 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:17.314663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:17.378136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 577/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:17.644702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270723c4-1c0e-4828-9317-e59ec7b1e07f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:17.743498 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270723c4-1c0e-4828-9317-e59ec7b1e07f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:17.780167 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f07fef-ea43-4122-a5db-91051ec84150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:17.780788 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07fef-ea43-4122-a5db-91051ec84150 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:17.781070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07fef-ea43-4122-a5db-91051ec84150 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:17.781347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f07fef-ea43-4122-a5db-91051ec84150 None admin] PATH_INFO: `/v3/domains/d99e90aa994748b298aa43195eeaf26d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:17.782455 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f07fef-ea43-4122-a5db-91051ec84150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d99e90aa994748b298aa43195eeaf26d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:17.783910 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f07fef-ea43-4122-a5db-91051ec84150 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:17.785458 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 599/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Fri Sep 16 19:07:17 2022] GET /identity/v3/domains/d99e90aa994748b298aa43195eeaf26d => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:17.791202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:17.885454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa93ca-4fcb-4a31-9851-60a8aa17f4f4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:17.904219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:17.918260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcbce5-e440-4246-be70-9a59d273a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:17.918851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcbce5-e440-4246-be70-9a59d273a94f None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:17.919123 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcbce5-e440-4246-be70-9a59d273a94f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:17.919423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcbce5-e440-4246-be70-9a59d273a94f None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:17.920054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcbce5-e440-4246-be70-9a59d273a94f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:17.921470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcbce5-e440-4246-be70-9a59d273a94f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:17.943122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 578/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:07:17 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:07:20.084653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b39ec6-3fdb-4fdf-b222-1d9994a2694c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:20.085048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b39ec6-3fdb-4fdf-b222-1d9994a2694c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:20.085201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b39ec6-3fdb-4fdf-b222-1d9994a2694c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:20.086526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 600/1178] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:20.091932 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:20.092359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:20.092482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:20.110740 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:20.111949 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:20.175442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 579/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:20.443245 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b39ec6-3fdb-4fdf-b222-1d9994a2694c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:20.497645 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:20.541502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b39ec6-3fdb-4fdf-b222-1d9994a2694c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:20.580511 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:20.581689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:20.581962 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:20.582229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:20.592311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:07:20.597048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:07:20.598149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:07:20.599220 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 601/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:07:20.604312 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:20.700339 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a2b14-2ecb-4c8c-97f6-c45cae12f4e6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:20.708404 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:20.737111 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:20.737713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:20.737993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:20.738265 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:20.738925 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:20.739228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:20.740867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:20.748682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 580/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:20.754684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:20.849842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0149c-6ceb-458f-b708-3e77fe2a78d6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:20.862535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:20.882179 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:20.882799 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:20.883077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:20.883379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] PATH_INFO: `/v3/users/a88f50dcc18849388b3284f005d330c8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:20.897163 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=a88f50dcc18849388b3284f005d330c8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:20.898600 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:20.900244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 602/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:07:20 2022] GET /identity/v3/users/a88f50dcc18849388b3284f005d330c8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:20.906372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:21.008754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c04303-ae8b-448d-973b-98dd48c4b99e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:21.041350 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:21.041942 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:21.042216 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:21.042484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:21.074835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:21.106740 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:21.108209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:21.200682 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 581/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:07:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:07:21.207167 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:21.303100 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12f49-d46a-4f30-805d-6bb5f283e0b0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:21.338036 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9772674-b39d-48e7-a100-6e18a5bee709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:21.338646 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9772674-b39d-48e7-a100-6e18a5bee709 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:21.338922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9772674-b39d-48e7-a100-6e18a5bee709 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:21.339191 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9772674-b39d-48e7-a100-6e18a5bee709 None admin] PATH_INFO: `/v3/projects/4388595b11564237a5394cf0e825c1a2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:21.340307 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9772674-b39d-48e7-a100-6e18a5bee709 None admin] RBAC: Authorizing `identity:get_project(project_id=4388595b11564237a5394cf0e825c1a2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:21.341724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9772674-b39d-48e7-a100-6e18a5bee709 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:21.343334 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 603/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:07:21 2022] GET /identity/v3/projects/4388595b11564237a5394cf0e825c1a2 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:21.349490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:21.412188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:21.444584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a49b-b43f-4153-9b45-a501cb0444b1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:21.476259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:21.476842 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:21.477116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:21.477384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:21.478088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=4388595b11564237a5394cf0e825c1a2, user.id=a88f50dcc18849388b3284f005d330c8) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:21.478389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:21.479812 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97315605-6281-481e-858b-1dc62d2fc8ce None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:21.484858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 582/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:07:21 2022] GET /identity/v3/role_assignments?user.id=a88f50dcc18849388b3284f005d330c8&scope.project.id=4388595b11564237a5394cf0e825c1a2&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:23.578393 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40e7f-c16c-4607-8a5c-11f655d2a2af None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:23.578960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40e7f-c16c-4607-8a5c-11f655d2a2af None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:23.578960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb40e7f-c16c-4607-8a5c-11f655d2a2af None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:23.580464 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 604/1186] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:23.586343 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:23.586763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:23.586867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:23.604477 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:23.605633 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:23.668583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 583/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:23.943266 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40e7f-c16c-4607-8a5c-11f655d2a2af None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:23.985343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:24.049059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb40e7f-c16c-4607-8a5c-11f655d2a2af None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:24.082367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:24.082985 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:24.083257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:24.083552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:24.093136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:07:24.097859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:07:24.098958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:07:24.100092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 605/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:07:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:07:24.104976 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:24.162947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:24.212424 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3577d-ccc3-43fd-83ce-9801b3a983e0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:24.244920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:24.245520 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:24.245799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:24.246069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:24.246749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:24.247046 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:24.248430 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:24.256591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 584/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:07:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:24.263059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:24.358331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:24.361285 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60c59-cad0-4e7f-9186-825d2cb7fc4b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:24.395864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:24.396478 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:24.396755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:24.397025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] PATH_INFO: `/v3/users/a88f50dcc18849388b3284f005d330c8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:24.398145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] RBAC: Authorizing `identity:get_user(user_id=a88f50dcc18849388b3284f005d330c8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:24.399584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:24.401217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 606/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:07:24 2022] GET /identity/v3/users/a88f50dcc18849388b3284f005d330c8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:24.407055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:24.504772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26240a-844c-49a9-9991-efc70017da4e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:24.526475 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:24.537114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a1d55-f71b-42ca-8d4c-cf7a74b57fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:24.537733 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a1d55-f71b-42ca-8d4c-cf7a74b57fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:24.538014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a1d55-f71b-42ca-8d4c-cf7a74b57fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:24.538289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a1d55-f71b-42ca-8d4c-cf7a74b57fa9 None admin] PATH_INFO: `/v3/projects/4388595b11564237a5394cf0e825c1a2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:24.539491 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a1d55-f71b-42ca-8d4c-cf7a74b57fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=4388595b11564237a5394cf0e825c1a2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:24.540920 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a1d55-f71b-42ca-8d4c-cf7a74b57fa9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:24.542506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 585/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:07:24 2022] GET /identity/v3/projects/4388595b11564237a5394cf0e825c1a2 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:24.548488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:24.650898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ce211-c34b-4f72-bd3e-5a5ada36275a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:24.685065 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcf8ab-8180-4775-a8ed-33ab46c3106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:24.685711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf8ab-8180-4775-a8ed-33ab46c3106c None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:24.685990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf8ab-8180-4775-a8ed-33ab46c3106c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:24.686261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf8ab-8180-4775-a8ed-33ab46c3106c None admin] PATH_INFO: `/v3/projects/4388595b11564237a5394cf0e825c1a2/users/a88f50dcc18849388b3284f005d330c8/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:24.688165 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcf8ab-8180-4775-a8ed-33ab46c3106c None admin] RBAC: Authorizing `identity:create_grant(project_id=4388595b11564237a5394cf0e825c1a2, user_id=a88f50dcc18849388b3284f005d330c8, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:24.689581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcf8ab-8180-4775-a8ed-33ab46c3106c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:24.701773 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 607/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:07:24 2022] PUT /identity/v3/projects/4388595b11564237a5394cf0e825c1a2/users/a88f50dcc18849388b3284f005d330c8/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:07:26.839864 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f5036-9136-41b4-955b-380ea038e0b0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:26.840431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f5036-9136-41b4-955b-380ea038e0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:26.840431 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f5036-9136-41b4-955b-380ea038e0b0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:26.841754 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 586/1193] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:07:26.847675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:26.848139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:26.848239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:26.866089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:07:26.867262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:07:26.888759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:26.932337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 608/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:07:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:07:27.199115 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f5036-9136-41b4-955b-380ea038e0b0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:27.296463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f5036-9136-41b4-955b-380ea038e0b0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:27.328154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:27.328763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:27.329039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:27.329309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:27.339795 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:07:27.344561 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:07:27.345643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:07:27.347316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 587/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:07:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:07:27.352254 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:27.446401 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f81c5-c3bd-4471-953c-93334252ca84 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:27.479320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:27.479868 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:27.480146 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:27.480418 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:27.481069 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:27.481369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:27.482717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:27.490627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 609/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:07:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:27.496399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:27.547032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:27.591661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55658d0-8e41-4752-af68-fdedab7afb71 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:27.623824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:27.624435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:27.624712 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:27.624984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] PATH_INFO: `/v3/users/a88f50dcc18849388b3284f005d330c8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:27.626074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] RBAC: Authorizing `identity:get_user(user_id=a88f50dcc18849388b3284f005d330c8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:27.627547 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:27.629078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 588/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Fri Sep 16 19:07:27 2022] GET /identity/v3/users/a88f50dcc18849388b3284f005d330c8 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:27.634636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:27.656388 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:27.731345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2f44e-52d0-4589-b5fa-7f8e6626b503 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:27.762849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:27.763463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:27.763749 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:27.764020 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:27.827894 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:27.829313 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:27.848977 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:27.922890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 610/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:07:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:07:27.928806 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:28.024542 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff235ab2-a350-4831-8a3e-80a139fae91f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:28.042886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:28.056420 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f0a8-3acc-4747-8d18-4fbea9613bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:28.057034 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f0a8-3acc-4747-8d18-4fbea9613bcb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:28.057315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f0a8-3acc-4747-8d18-4fbea9613bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:28.057588 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f0a8-3acc-4747-8d18-4fbea9613bcb None admin] PATH_INFO: `/v3/projects/4388595b11564237a5394cf0e825c1a2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:28.058697 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f0a8-3acc-4747-8d18-4fbea9613bcb None admin] RBAC: Authorizing `identity:get_project(project_id=4388595b11564237a5394cf0e825c1a2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:28.060139 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f0a8-3acc-4747-8d18-4fbea9613bcb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:28.061762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 589/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Fri Sep 16 19:07:27 2022] GET /identity/v3/projects/4388595b11564237a5394cf0e825c1a2 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:28.068262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:07:28.163752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed50af-1eff-49de-9763-a32367abb738 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:07:28.183059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:28.197351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:07:28.197944 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:07:28.198224 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:07:28.198497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:07:28.199211 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=4388595b11564237a5394cf0e825c1a2, user.id=a88f50dcc18849388b3284f005d330c8) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:07:28.199553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:07:28.201006 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dbf8e-0b41-4e02-9283-bf1e1f24b97d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:07:28.206506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 611/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:07:28 2022] GET /identity/v3/role_assignments?user.id=a88f50dcc18849388b3284f005d330c8&scope.project.id=4388595b11564237a5394cf0e825c1a2&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:07:38.052448 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:07:38.204004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:23.942349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150fb4-e79f-44e9-9113-365c827011fd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:23.942349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150fb4-e79f-44e9-9113-365c827011fd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:23.942349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87150fb4-e79f-44e9-9113-365c827011fd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:23.942349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 590/1201] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:08:23.957764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:23.958494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:23.958494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:23.981379 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:23.982910 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:24.083328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 612/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 19:08:24.402879 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150fb4-e79f-44e9-9113-365c827011fd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:24.541035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87150fb4-e79f-44e9-9113-365c827011fd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:24.581757 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:24.582577 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:24.582983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:24.583405 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:24.596755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:24.598315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:24.618053 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:24.619566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:24.620723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:24.620723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:24.620723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:24.620723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 591/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:24.627325 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:24.755436 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce80b2-2419-4145-9db9-919110c5f15f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:24.804287 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:24.812318 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:24.813096 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:24.813534 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:24.814115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:24.814926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:24.815579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:24.817363 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028c975-8c1a-4c2b-8b9b-87f9390dc0c3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:24.835465 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 613/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:24.838166 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:24.910179 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:24.967210 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b04e3a-e02b-4111-bd21-c3fb31737f04 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:25.012777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80f2d4-b781-42b3-9a88-0627cec62795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:25.013328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80f2d4-b781-42b3-9a88-0627cec62795 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:25.013611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80f2d4-b781-42b3-9a88-0627cec62795 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:25.013886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80f2d4-b781-42b3-9a88-0627cec62795 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:25.014550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80f2d4-b781-42b3-9a88-0627cec62795 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:25.016075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80f2d4-b781-42b3-9a88-0627cec62795 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:25.051554 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 592/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Fri Sep 16 19:08:24 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:08:27.589049 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cad4-db1b-4e30-85c8-e040ca11a0dd None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:27.589621 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cad4-db1b-4e30-85c8-e040ca11a0dd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:27.589621 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469cad4-db1b-4e30-85c8-e040ca11a0dd None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:27.590994 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 614/1206] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:27.596662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:27.597142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:27.597244 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:27.617371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:27.618637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:27.702440 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 593/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:28.000557 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cad4-db1b-4e30-85c8-e040ca11a0dd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:28.114842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469cad4-db1b-4e30-85c8-e040ca11a0dd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:28.152521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:28.153398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:28.153770 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:28.154139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:28.166540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:08:28.171582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:28.172884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:08:28.174057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:28.183331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 615/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:08:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:28.183494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:28.236080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:28.304935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61efd69-c7ae-4f17-b233-f10d16263228 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:28.344443 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:28.345082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:28.345391 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:28.345781 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:28.346578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:28.346910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:28.348459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:28.363101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 594/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:28.365102 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:28.419274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:28.481835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d05a2-8788-46d2-b5c7-daf004a3d653 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:28.528723 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:28.529398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:28.529681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:28.529957 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:28.543762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:28.545426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:28.562582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:28.569261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:28.570196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:28.570196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:28.570196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:28.570196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 616/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:28.570435 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:28.679020 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:28.683418 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3477e4a-7177-4156-b0ed-7eee7baa004e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:28.722733 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:28.723411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:28.723688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:28.723963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:28.724637 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:28.724927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:28.726285 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:28.746235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 595/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:28.749781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:28.865724 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b83db-3b2d-4069-8773-29aab25a37bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:28.867273 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:28.901964 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:28.902567 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:28.902845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:28.903142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:28.911656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:28.913139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:08:28.926233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:28.927437 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:28.928546 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 617/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:08:28.933247 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:29.037321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f649b-a156-4f76-8cc4-31603798aa5a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:29.080332 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:29.081032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:29.081335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:29.081652 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:29.082388 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:29.082726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:29.084242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:29.101727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 596/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Sep 16 19:08:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:29.107373 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:29.241213 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5321-8064-49de-8875-e33fb02eb5a2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:29.283848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:29.284539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:29.284860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:29.285164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:29.318598 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:29.359855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:29.361581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:29.474261 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 618/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:08:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:08:29.480059 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:29.502479 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:29.622857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aff95-1dca-4677-8ace-1cf6d393a5c4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:29.665662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:29.666370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:29.666713 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:29.667051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:29.679782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:29.681425 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:29.700121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:29.701452 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:29.702522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:29.702522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:29.702522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:29.702522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 597/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:29.708194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:29.778343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:29.818735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341113a-2616-4a4d-a6de-3059857fd418 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:29.856842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:29.857528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:29.857873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:29.858207 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:29.858972 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:29.859311 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:29.860908 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:29.873657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 619/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:29.879718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:29.956171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:29.994790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d826-b2d5-4777-a194-1c7fc79fa529 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:30.032835 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:30.033395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:30.033679 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:30.033956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:30.040484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:30.041994 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:08:30.054433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:30.055604 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:30.056530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 598/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:30.061816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:30.167945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a01846-b1c9-4094-865a-cdacd218baa3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:30.193604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:30.206834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:30.207552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:30.207824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:30.208120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:30.208979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:30.209286 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:30.210831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097efec-b7e6-4ddf-8830-f04d2687cc24 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:30.225748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 620/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:30.232200 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:30.338966 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ea7e2-1fdd-47c0-94e9-b67ba96602d0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:30.369519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:30.377181 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:30.377870 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:30.378150 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:30.378421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:30.379165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=60a4863b078c436ab224120b8c491f50) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:30.379530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:30.381262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9de518-2783-44e5-8520-9dc9326fde2c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:30.386953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 599/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:08:30 2022] GET /identity/v3/role_assignments?user.id=60a4863b078c436ab224120b8c491f50&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:32.611702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162e6ed-32d6-4fcc-963f-fdfc43fc797f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:32.612231 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162e6ed-32d6-4fcc-963f-fdfc43fc797f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:32.612347 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162e6ed-32d6-4fcc-963f-fdfc43fc797f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:32.613841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 621/1220] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:32.620515 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:32.620944 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:32.621031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:32.643160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:32.644695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:32.717952 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 600/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:33.041187 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162e6ed-32d6-4fcc-963f-fdfc43fc797f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:33.166583 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162e6ed-32d6-4fcc-963f-fdfc43fc797f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:33.202083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:33.202689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:33.202970 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:33.203246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:33.214568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:08:33.219416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:33.220541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:33.221559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 622/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:33.227118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:33.329935 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce464f-a49c-4836-9374-d656b549d24a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:33.356855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:33.368783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:33.369565 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:33.369991 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:33.370428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:33.371228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:33.371719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:33.373231 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:33.381678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 601/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:08:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:33.387693 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:33.435965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:33.494315 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0ceb2-f502-4430-93ca-e9501cb51ad9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:33.528879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:33.529507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:33.529850 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:33.530057 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:33.540120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:33.541547 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:33.558288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:33.559558 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:33.564883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:33.564883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:33.564883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:33.564883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 623/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:33.566289 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:33.671538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bba56-b306-4f0a-96ad-31c585bfcfd4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:33.709321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:33.709953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:33.710232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:33.710528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:33.711167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:33.711501 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:33.712909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:33.733638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 602/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:33.736095 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:33.842073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:33.850271 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73935ada-c991-4b22-ae35-4e35d325aad9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:33.888548 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:33.889167 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:33.889443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:33.889712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:33.897017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:33.898438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:08:33.912351 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:33.913494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:33.914477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 624/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:33.920041 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:33.973219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:34.031018 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b2e9e-356b-4229-8014-0da95a9a3bca None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:34.068963 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:34.069583 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:34.069909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:34.070257 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:34.071159 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:34.071564 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:34.073151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:34.094414 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 603/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Sep 16 19:08:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:34.100531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:34.223073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52833d1a-0536-4f52-a3df-2ed649d83704 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:34.261360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:34.262080 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:34.262362 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:34.262722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:34.275926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:34.277449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:34.295510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:34.296841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:34.298618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:34.298618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:34.298618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:34.298618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 625/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:34.303525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:34.416094 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f714-c990-4d66-a67a-b5fdb8062146 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:34.419007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:34.451766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:34.452402 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:34.452687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:34.452960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:34.454114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:34.454417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:34.455867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:34.467548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 604/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:34.474126 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:34.484592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:34.580241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3309e-f462-48fe-9882-874925622a64 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:34.617411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:34.618048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:34.618360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:34.618685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:34.626505 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:34.627980 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:08:34.639359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:34.640697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:34.642550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 626/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:08:34.647321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:34.785168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee6805-c1d0-4236-a9ed-d82de54309ae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:34.824080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:34.824688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:34.824971 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:34.825246 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:34.825930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:34.826234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:34.827687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd1665-a795-42f9-8eaf-e9208ab18c76 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:34.839796 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 605/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:34.846660 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:34.914209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:34.950022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d01d30-8f6b-4c40-a7c3-030e6dbc6bc4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:34.985072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc15e22-80cc-44f4-9d35-730831c2962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:34.985725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc15e22-80cc-44f4-9d35-730831c2962d None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:34.986004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc15e22-80cc-44f4-9d35-730831c2962d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:34.986280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc15e22-80cc-44f4-9d35-730831c2962d None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/60a4863b078c436ab224120b8c491f50/roles/6672f082d46146fabfbd0db1bc2c62ca` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:35.001330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc15e22-80cc-44f4-9d35-730831c2962d None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=60a4863b078c436ab224120b8c491f50, role_id=6672f082d46146fabfbd0db1bc2c62ca)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:35.002895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc15e22-80cc-44f4-9d35-730831c2962d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:35.015608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 627/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:08:34 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/60a4863b078c436ab224120b8c491f50/roles/6672f082d46146fabfbd0db1bc2c62ca => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:08:37.225352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f20b8-a90a-4443-86ee-0f72dea75607 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:37.226010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f20b8-a90a-4443-86ee-0f72dea75607 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:37.226010 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f20b8-a90a-4443-86ee-0f72dea75607 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:37.227372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 606/1233] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:37.232711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a92c7-9070-47cf-887f-a94589062504 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:37.233227 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a92c7-9070-47cf-887f-a94589062504 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:37.233227 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a92c7-9070-47cf-887f-a94589062504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:37.254650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-209a92c7-9070-47cf-887f-a94589062504 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:37.255883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-209a92c7-9070-47cf-887f-a94589062504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:37.327042 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 628/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:37.612400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f20b8-a90a-4443-86ee-0f72dea75607 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:37.643659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:37.735521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f20b8-a90a-4443-86ee-0f72dea75607 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:37.789316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:37.790104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:37.790384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:37.790710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:37.802643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:08:37.807869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:37.809127 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 19:08:37.810142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:37.814454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 607/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Fri Sep 16 19:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:37.815796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a92c7-9070-47cf-887f-a94589062504 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:37.868986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:37.927118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a92c7-9070-47cf-887f-a94589062504 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:37.967267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:37.967897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:37.968177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:37.968451 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:37.969115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:37.970000 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:37.971386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:37.979540 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 629/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Fri Sep 16 19:08:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:37.987349 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:38.083594 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:38.101069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a12d6-8cc2-442a-8c68-372260736c21 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:38.137804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:38.138471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:38.138858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:38.139269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:38.152243 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:38.154395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:38.170687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:38.171883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:38.172855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:38.172855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:38.172855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:38.172855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 608/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:38.178195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:38.286253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d6c9f-5e5b-49b5-a0b0-7a6c4bcc4174 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:38.295257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:38.321933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:38.322551 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:38.322818 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:38.323086 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:38.323778 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:38.324064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:38.325484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:38.339036 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 630/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:38.346007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:38.497107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664d7b5-5ba8-4eb4-90b1-aa2b71c88c2d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:38.540690 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:38.541291 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:38.541616 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:38.541893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:38.545020 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:38.552862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:38.554641 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:08:38.575211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:38.576400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:38.577375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 609/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:38.587372 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:38.704975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2af6f9-7754-4117-aea4-b31d19e33eb6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:38.744247 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:38.744924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:38.745196 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:38.745472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:38.746206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:38.746513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:38.748019 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:38.768445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 631/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Sep 16 19:08:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:38.775691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:38.886453 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25381acd-f89e-4eac-9b91-89ec17784dde None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:38.925939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:38.926708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:38.927040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:38.927374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:39.002254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:39.009663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:39.011508 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:39.120854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 610/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:08:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:08:39.127657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:39.157181 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:39.235787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5816-4118-41e8-8504-fa6d309ae2bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:39.272183 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:39.272710 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:39.273032 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:39.273253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:39.283402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:39.284744 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:39.301212 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:39.302845 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:39.304073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:39.304073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:39.304073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:39.307419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 632/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:39.308603 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:39.408977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:39.415412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a29d6-4761-4732-9335-f29c69938b8a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:39.450975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:39.451651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:39.451975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:39.452267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:39.452981 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:39.453321 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:39.455199 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:39.474160 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 611/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:39.475678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:39.562440 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:39.586284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cbad9-efe9-4a03-b7a0-67f5604d397e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:39.622875 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:39.623719 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:39.624003 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:39.624288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:39.631267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:39.632815 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:08:39.645381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:39.646445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:39.647399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 633/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:39.652613 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:39.763037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb43fff-06bd-4e48-9b2b-1c6312adc08e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:39.807749 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:39.808367 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:39.808658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:39.808931 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:39.809716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:39.810045 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:39.811591 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bad2b7-e642-4e70-a8f3-9945d97ee579 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:39.826268 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 612/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:39.834378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:39.944705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abff62-67d2-41d1-8d34-0b39523e94ac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:39.981789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:39.982485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:39.982766 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:39.983142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:39.983819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] RBAC: Adding query filter params (role.id=6672f082d46146fabfbd0db1bc2c62ca, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=60a4863b078c436ab224120b8c491f50) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:39.984124 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:39.985776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2e623-f9e7-4e6d-88eb-0b2290721cf7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:39.994841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 634/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:08:39 2022] GET /identity/v3/role_assignments?user.id=60a4863b078c436ab224120b8c491f50&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=6672f082d46146fabfbd0db1bc2c62ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:42.217677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0305c-d51a-4f20-8c78-93dbb6d746fa None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:42.218228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0305c-d51a-4f20-8c78-93dbb6d746fa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:42.218228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0305c-d51a-4f20-8c78-93dbb6d746fa None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:42.220131 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 613/1247] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:42.225471 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:42.225878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:42.226010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:42.232480 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:42.246386 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:42.247620 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:42.317986 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 635/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:42.668262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0305c-d51a-4f20-8c78-93dbb6d746fa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:42.688177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:42.788581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0305c-d51a-4f20-8c78-93dbb6d746fa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:42.824649 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:42.825331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:42.825612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:42.825884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:42.837036 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:08:42.841889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:42.843022 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:42.844375 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 614/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:42.849201 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:42.955323 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cb5d-58fe-4b8f-a5c8-fc1da8caa00e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:42.987847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:42.993630 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:42.994254 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:42.994529 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:42.994796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:42.995503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:42.995804 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:42.997195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:43.006512 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 636/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:08:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:43.012954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:43.095082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:43.119880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddf594-0ae9-4aa9-a34f-8ac84e44655f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:43.155851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:43.157080 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:43.157359 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:43.157631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:43.168262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:43.169723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:43.186155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:43.187168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:43.188236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:43.188236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:43.188236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:43.188236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 615/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 19:08:43.193369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:43.296708 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ebd3c-ca31-4e6c-929f-daa8d4b4ba7c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:43.329716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:43.330321 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:43.330655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:43.330932 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:43.331604 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:43.331910 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:43.333260 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:43.345568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 637/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 19:08:43.351528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:43.456832 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763ac0a-78c8-4877-95e0-0a9ded9ac114 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:43.483073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:43.491888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:43.492525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:43.492809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:43.493079 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:43.500770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:43.502276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:08:43.513915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:43.514889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:43.516429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 616/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:43.521446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:43.577435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:43.623826 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc783-fe44-4d9f-918c-b6a1445f9a63 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:43.658678 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:43.659282 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:43.659590 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:43.659862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:43.660577 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:43.660882 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:43.662297 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:43.679224 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 638/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Sep 16 19:08:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:43.685118 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:43.792050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f71de-ed46-4161-a431-599d8b0ca711 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:43.824764 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:43.825360 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:43.825639 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:43.825910 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:43.878706 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:43.890701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:43.892771 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:43.986101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 617/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:08:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:08:43.992136 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:44.091381 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc52f64-e5ca-41a5-b683-18e0e4733629 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:44.104769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:44.126499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:44.127117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:44.127428 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:44.127703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:44.137285 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:44.138647 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:44.152592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:44.153559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:44.154709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:44.154709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:44.154709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:44.154709 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 639/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:44.159394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:44.242923 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:44.255464 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f8bf-ebed-441b-9046-f7f8382c1617 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:44.288116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:44.288708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:44.288987 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:44.289259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:44.289889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:44.290188 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:44.291567 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:44.302958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 618/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:44.308567 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:44.403303 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce296a-b579-46a5-855e-ed0b4d6e23e6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:44.435696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:44.436299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:44.436574 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:44.436841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:44.443174 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:44.444614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:08:44.455421 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:44.456525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:44.457760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 640/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:44.462292 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:44.559850 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a1487-8553-45e2-99b8-a85dcac68569 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:44.591207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:44.591824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:44.592103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:44.592377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:44.593054 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:44.593357 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:44.595342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6434-6bb7-45c0-abbb-5d56fcdaa5f7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:44.606699 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 619/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:44.613269 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:44.637954 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:44.710284 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7403c-26d3-4c80-a815-f0ebc6769986 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:44.749244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:44.749952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:44.750310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:44.750711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:44.751949 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=60a4863b078c436ab224120b8c491f50) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:44.752288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:44.754066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835514a5-96b7-4763-b7dd-f326432c3357 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:44.760902 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 641/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:08:44 2022] GET /identity/v3/role_assignments?user.id=60a4863b078c436ab224120b8c491f50&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:46.852294 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe0a45-806e-4895-a45f-e39923e47a6a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:46.852846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe0a45-806e-4895-a45f-e39923e47a6a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:46.852846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe0a45-806e-4895-a45f-e39923e47a6a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:46.854175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 620/1261] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:46.859304 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:46.860342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:46.860439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:46.878262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:46.879474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:46.942486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 642/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:47.212031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe0a45-806e-4895-a45f-e39923e47a6a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:47.223378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:47.310439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe0a45-806e-4895-a45f-e39923e47a6a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:47.343148 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:47.343838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:47.344122 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:47.344394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:47.353946 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:08:47.358710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:47.359653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:47.360509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 621/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:47.366139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:47.384320 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:47.462370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe57fe-5313-486d-b98a-7f24866feaad None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:47.494002 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:47.494591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:47.494865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:47.495139 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:47.495816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:47.496115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:47.497457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:47.505533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 643/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:47.511239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:47.576103 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:47.608596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6552c-bad2-4356-acf9-3607eb2f3c14 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:47.641940 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:47.642549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:47.642827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:47.643100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:47.652698 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:47.654125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:47.668219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:47.669279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:47.670506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:47.670506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:47.670506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:47.670506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 622/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:47.675545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:47.763800 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:47.783663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4717a-257a-48c0-a94d-58f85e6b8546 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:47.816413 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:47.817003 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:47.817277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:47.817545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:47.818164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:47.818457 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:47.819941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:47.831276 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 644/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:47.837035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:47.933411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cd93a-ccbd-404d-a1ad-da3e6de593fc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:47.952703 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:47.966983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:47.967640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:47.967922 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:47.968196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:47.975407 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:47.976830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:08:47.988167 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:47.989227 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:47.990370 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 623/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:47.995370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:48.091934 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d74054-7240-47ac-9541-81907fe42bbc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:48.125288 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:48.125880 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:48.126164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:48.126438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:48.127150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:48.127482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:48.128892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:48.137166 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:48.145592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 645/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Sep 16 19:08:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:48.151459 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:48.248047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011869-e862-42b9-a796-2f347fef74cd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:48.279423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:48.280031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:48.280310 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:48.280584 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:48.291163 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:48.292627 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:48.306705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:48.307840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:48.308741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:48.308741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:48.308741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:48.308741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 624/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:48.313343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:48.407834 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52567f51-34fa-4636-9ad9-3b704288c936 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:48.439830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:48.440419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:48.440697 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:48.440969 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:48.441594 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:48.441899 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:48.443243 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:48.454412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 646/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:48.460021 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:48.491705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:48.557318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c3e70-5b15-481f-951f-ca49d9e0651a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:48.589462 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:48.590078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:48.590354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:48.590624 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:48.597248 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:48.598677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:08:48.610025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:48.611055 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:48.612030 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 625/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:48.617267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:48.673821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:48.714503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d02a-e003-4797-9c80-1ad912c09508 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:48.757676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:48.758277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:48.758555 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:48.758831 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:48.759542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:48.759849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:48.761270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dde92c-7637-4814-af1f-3866c6d51d87 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:48.773415 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 647/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:08:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:48.779423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:48.867532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:48.876558 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de81555-ee3c-413b-992b-7da49fc54366 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:48.908513 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a370e1c-0df0-4d78-8c72-29b1a905fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:48.909146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370e1c-0df0-4d78-8c72-29b1a905fd80 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:48.909421 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370e1c-0df0-4d78-8c72-29b1a905fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:48.909690 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370e1c-0df0-4d78-8c72-29b1a905fd80 None admin] PATH_INFO: `/v3/projects/ad877dffc51743268f5541983c925a8d/users/60a4863b078c436ab224120b8c491f50/roles/2a68e1ac7d044097aa76bf9263f10225` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:48.911549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a370e1c-0df0-4d78-8c72-29b1a905fd80 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad877dffc51743268f5541983c925a8d, user_id=60a4863b078c436ab224120b8c491f50, role_id=2a68e1ac7d044097aa76bf9263f10225)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:48.912953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a370e1c-0df0-4d78-8c72-29b1a905fd80 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:48.924889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 626/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Fri Sep 16 19:08:48 2022] PUT /identity/v3/projects/ad877dffc51743268f5541983c925a8d/users/60a4863b078c436ab224120b8c491f50/roles/2a68e1ac7d044097aa76bf9263f10225 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:08:51.026159 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510d58-1017-4007-bd92-e431e2d3ee7b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:51.026712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510d58-1017-4007-bd92-e431e2d3ee7b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:51.026712 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510d58-1017-4007-bd92-e431e2d3ee7b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:51.028039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 648/1274] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:51.033968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:51.034364 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:51.034511 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:51.052387 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:51.053570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:51.118170 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 627/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:51.382718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e510d58-1017-4007-bd92-e431e2d3ee7b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:51.480448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e510d58-1017-4007-bd92-e431e2d3ee7b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:51.497510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:51.514752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:51.515398 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:51.515682 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:51.515959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:51.525650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 19:08:51.530426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:51.531539 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:51.532565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 649/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Fri Sep 16 19:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:51.537769 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:51.615177 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:51.635138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5676c-be0a-4132-b2f2-ecd1e4e00551 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:51.668201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:51.668791 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:51.669065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:51.669335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:51.669982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:51.670278 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:51.671653 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:51.679307 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 628/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Fri Sep 16 19:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:51.685063 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:51.792717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4388af6-d33b-403c-a0c8-4db9d3549568 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:51.825995 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:51.826612 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:51.826885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:51.827152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:51.837965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:51.839399 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:51.852925 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:51.853922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:51.854948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:51.854948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:51.854948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:51.854948 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 650/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:51.859796 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:51.957732 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee769cd-6fbb-4dab-b923-c5e90d0da4a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:51.989362 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:51.989956 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:51.990234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:51.990506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:51.991437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:51.991741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:51.993087 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:52.004802 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 629/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:52.010418 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:52.040681 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:52.107924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923d116-3800-4109-924a-bb1d21f5cc52 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:52.140189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:52.141397 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:52.141669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:52.141933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:52.149010 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:52.150416 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 19:08:52.161157 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:52.162257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:52.163654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 651/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:52.167993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:52.171715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:52.265881 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4807cbd-b4b3-45c5-9814-72aef6886988 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:52.297983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:52.298577 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:52.298851 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:52.299117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:52.299857 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:52.300158 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:52.301560 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:52.318159 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 630/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Fri Sep 16 19:08:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:52.323887 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:52.410355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:52.419808 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010e1a-bbf1-41e5-895e-8432ac94a7fe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:52.452395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:52.453033 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:52.453314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:52.453586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:52.517385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:52.518748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:52.602051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:52.612229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 652/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Fri Sep 16 19:08:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 19:08:52.618324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:52.678658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:52.716553 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41bb12-54c3-43ab-b8b1-ba8359ad379a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:52.750326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:52.750968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:52.751245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:52.751550 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:52.761129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:52.762552 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 19:08:52.777154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:52.778229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 19:08:52.779315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 19:08:52.779315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 19:08:52.779315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:52.779315 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 631/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:52.783909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:52.876973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62ca9e-951f-4d58-8c3c-ff5ff8c7c484 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:52.909367 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:52.909945 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:52.910217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:52.910486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:52.911102 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:52.911427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:52.912764 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:52.923827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 653/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Fri Sep 16 19:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:52.929334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:53.028272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea2ce4-5cdd-4f6f-84fc-f41ce9e58e4c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:53.041088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:53.061578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:53.062202 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:53.062538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:53.062817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:53.069423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:53.070858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 19:08:53.082538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:53.083688 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 19:08:53.084762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 16 19:08:53.085328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 632/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Fri Sep 16 19:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:53.091581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:53.142622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:53.189930 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c772e1a-a9bb-4981-b250-b08042980cd5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:53.221897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:53.222479 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:53.222752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:53.223019 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:53.223718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:53.224021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:53.225424 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944c626-125f-453f-8e29-5abd872767bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:53.236754 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 654/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Fri Sep 16 19:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:53.243465 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:53.339691 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782b3f6-b31c-4de7-821e-d6d632ad1af9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:53.375611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:53.376217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:53.376496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:53.376768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:53.377496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] RBAC: Adding query filter params (role.id=2a68e1ac7d044097aa76bf9263f10225, scope.project.id=ad877dffc51743268f5541983c925a8d, user.id=60a4863b078c436ab224120b8c491f50) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:53.377805 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:53.379229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a02421-9020-4d56-a386-b41a6848e64a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:53.385124 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 633/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Fri Sep 16 19:08:53 2022] GET /identity/v3/role_assignments?user.id=60a4863b078c436ab224120b8c491f50&scope.project.id=ad877dffc51743268f5541983c925a8d&role.id=2a68e1ac7d044097aa76bf9263f10225 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:55.562330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82848f-7a6e-45c1-9286-7cc1929b0700 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:55.562842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82848f-7a6e-45c1-9286-7cc1929b0700 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:55.562952 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82848f-7a6e-45c1-9286-7cc1929b0700 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:55.564233 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 655/1288] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:55.569279 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:55.569638 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:55.569817 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:55.587478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:55.588646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:55.617434 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:55.654039 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 634/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:55.949917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82848f-7a6e-45c1-9286-7cc1929b0700 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:56.051748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82848f-7a6e-45c1-9286-7cc1929b0700 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:56.062166 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:56.085588 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:56.086208 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:56.086488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:56.086757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:56.087383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:56.088735 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:08:56.097883 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:08:56.099448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:08:56.100641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 656/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:08:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:08:56.104886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:56.197442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1e34-2ec9-4ab9-823d-b3f2f93378ac None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:56.230145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:56.230669 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:56.230943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:56.231211 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:56.231843 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:56.232139 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:56.233491 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d003ea-1b26-4bbc-86c3-08e34b4dc929 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:56.238050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 635/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Sep 16 19:08:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:56.243865 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:56.339728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751f7e8-60ce-4178-b028-b41dbfb0181a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:56.371152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:56.371758 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:56.372034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:56.372303 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:56.372909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:08:56.373206 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:56.374544 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bed376-56ac-49b1-9950-bbd7523a78f2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:56.379217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 657/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Sep 16 19:08:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:08:58.520940 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7157-e00a-4783-ba1b-6afd940f428a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:58.521487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7157-e00a-4783-ba1b-6afd940f428a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:58.521487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7157-e00a-4783-ba1b-6afd940f428a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:58.522815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 636/1293] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:08:58.527878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcbbb7-ff11-4177-ae76-98d2e708464a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:58.528293 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcbbb7-ff11-4177-ae76-98d2e708464a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:58.528410 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcbbb7-ff11-4177-ae76-98d2e708464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:58.546501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4bfcbbb7-ff11-4177-ae76-98d2e708464a None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:08:58.547733 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4bfcbbb7-ff11-4177-ae76-98d2e708464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:08:58.576384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:58.612469 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 658/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:08:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:08:58.876889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad7157-e00a-4783-ba1b-6afd940f428a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:08:58.906884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:08:58.974429 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad7157-e00a-4783-ba1b-6afd940f428a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:08:59.007496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6427e-da80-42ac-bcd0-52fa4ac3b298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:08:59.008087 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6427e-da80-42ac-bcd0-52fa4ac3b298 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:08:59.008360 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6427e-da80-42ac-bcd0-52fa4ac3b298 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:08:59.008624 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6427e-da80-42ac-bcd0-52fa4ac3b298 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:08:59.009264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6427e-da80-42ac-bcd0-52fa4ac3b298 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:08:59.010607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6427e-da80-42ac-bcd0-52fa4ac3b298 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:08:59.024415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 637/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Fri Sep 16 19:08:58 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:09:01.168947 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff715ab-322f-436b-8253-b7337c9e758a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:01.169447 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff715ab-322f-436b-8253-b7337c9e758a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:01.169566 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff715ab-322f-436b-8253-b7337c9e758a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:01.170796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 659/1296] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:09:01.176399 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:01.176800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:01.176954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:01.195316 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:09:01.196510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:09:01.266037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 638/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:09:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:09:01.559365 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff715ab-322f-436b-8253-b7337c9e758a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:01.656896 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff715ab-322f-436b-8253-b7337c9e758a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:01.690352 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:01.690960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:01.691236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:01.691531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:01.692096 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:01.693440 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:09:01.701802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:09:01.702933 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:09:01.703881 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 660/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:09:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:09:01.708755 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:01.727466 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:09:01.811379 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a080002-98ad-4d61-a5d3-a066daf7b6be None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:01.848540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:01.849142 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:01.849422 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:01.849692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:01.850320 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:09:01.850621 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:01.852006 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:01.857116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 639/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Sep 16 19:09:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:09:01.863805 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:01.894836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:09:01.960748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db34b2a-13dc-4f22-9d1f-21e6a2252b2b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:01.993025 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:01.993610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:01.993884 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:01.994153 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:01.994826 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] RBAC: Adding query filter params (interface=public, service_id=d0ff8c5d0ff3400e92f0b48cb3bb01cf, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:09:01.995125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:01.996492 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31657-8c83-41e2-8467-f4a59d8b6982 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:02.001243 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 661/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Fri Sep 16 19:09:01 2022] GET /identity/v3/endpoints?service_id=d0ff8c5d0ff3400e92f0b48cb3bb01cf&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:09:02.006884 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:02.086222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:09:02.104439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d91b0-45e8-48fc-809d-2f6186ded698 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:02.136562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e77d6-5719-4ed6-b734-a87efd43e52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:02.137185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e77d6-5719-4ed6-b734-a87efd43e52b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:02.137463 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e77d6-5719-4ed6-b734-a87efd43e52b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:02.137736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e77d6-5719-4ed6-b734-a87efd43e52b None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:02.138328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e77d6-5719-4ed6-b734-a87efd43e52b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:02.139707 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e77d6-5719-4ed6-b734-a87efd43e52b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:02.146839 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 640/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Fri Sep 16 19:09:02 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 19:09:04.248128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eee43-97d1-4006-859a-2f0bd5519225 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:04.248694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eee43-97d1-4006-859a-2f0bd5519225 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:04.248694 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eee43-97d1-4006-859a-2f0bd5519225 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:04.250021 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 662/1302] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:09:04.255096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:04.255569 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:04.255684 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:04.273921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:09:04.275098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:09:04.339607 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 641/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:09:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 19:09:04.611005 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eee43-97d1-4006-859a-2f0bd5519225 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:04.702394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:09:04.709228 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eee43-97d1-4006-859a-2f0bd5519225 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:04.746361 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:04.746984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:04.747258 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:04.747568 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:04.748128 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:04.749487 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 19:09:04.760229 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 19:09:04.761528 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 16 19:09:04.762473 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 663/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Fri Sep 16 19:09:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 19:09:04.768339 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:04.842425 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:09:04.865599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12305dee-a2f4-4867-a18c-c051f5591416 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:04.897129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:04.897720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:04.897993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:04.898260 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:04.898868 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:09:04.899161 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:04.900533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68bd6e-94e2-4ccf-8006-f8bc53fa02aa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:04.905381 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 642/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Fri Sep 16 19:09:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:09:04.911655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:09:05.007812 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aeb39-10a8-414f-b54f-545afd8b4523 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:09:05.040759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cec627-c4a0-436f-a5ed-403f6540ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:09:05.041338 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec627-c4a0-436f-a5ed-403f6540ddac None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:09:05.041612 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec627-c4a0-436f-a5ed-403f6540ddac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:09:05.041878 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec627-c4a0-436f-a5ed-403f6540ddac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:09:05.042531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cec627-c4a0-436f-a5ed-403f6540ddac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:09:05.043893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cec627-c4a0-436f-a5ed-403f6540ddac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:09:05.062127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 664/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:09:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:09:14.902677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=30 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:09:15.058443 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:08.929045 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765e7ff-ec54-4fbc-8714-d08c581b0c71 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:08.929532 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765e7ff-ec54-4fbc-8714-d08c581b0c71 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:08.929622 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765e7ff-ec54-4fbc-8714-d08c581b0c71 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:08.931154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 643/1307] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:08.938746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902ee1b-01c5-4367-aa62-99a14f802413 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:08.939765 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902ee1b-01c5-4367-aa62-99a14f802413 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:08.940185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902ee1b-01c5-4367-aa62-99a14f802413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:09.087702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a902ee1b-01c5-4367-aa62-99a14f802413 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:09.089193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a902ee1b-01c5-4367-aa62-99a14f802413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:09.220606 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 665/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:09.516355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbef61e-4a86-4b4d-84a7-c6ff2db44b2a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:09.516828 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbef61e-4a86-4b4d-84a7-c6ff2db44b2a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:09.517232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbef61e-4a86-4b4d-84a7-c6ff2db44b2a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:09.518790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 644/1309] 10.222.0.18 () {54 vars in 963 bytes} [Fri Sep 16 19:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:09.525343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bed0c3-1b16-4dfd-b5e2-ec928489033f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:09.525770 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bed0c3-1b16-4dfd-b5e2-ec928489033f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:09.526152 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bed0c3-1b16-4dfd-b5e2-ec928489033f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:09.609862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-27bed0c3-1b16-4dfd-b5e2-ec928489033f None None] MFA Rules not processed for user `fc1c6323c0a34d8f9d36fdc95712d26f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:09.611073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-27bed0c3-1b16-4dfd-b5e2-ec928489033f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:09.632556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:09.692054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 666/1310] 10.222.0.18 () {58 vars in 1060 bytes} [Fri Sep 16 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:10:09.701414 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbef61e-4a86-4b4d-84a7-c6ff2db44b2a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:09.703156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4dbef61e-4a86-4b4d-84a7-c6ff2db44b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:09.814898 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbef61e-4a86-4b4d-84a7-c6ff2db44b2a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:09.893613 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:09.894219 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:09.894492 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:09.894761 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:09.896371 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:09.952911 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:10.009001 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:10.010487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2434-4ee4-4f23-b547-e066541e1858 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:10.029645 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 645/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:12.516739 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507538a-f846-45ad-970c-2253623fdc0a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:12.517355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507538a-f846-45ad-970c-2253623fdc0a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:12.517355 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507538a-f846-45ad-970c-2253623fdc0a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:12.518632 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 667/1312] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:12.523947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d426e3c-8522-423e-9691-d452997a4a7b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:12.524378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d426e3c-8522-423e-9691-d452997a4a7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:12.524482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d426e3c-8522-423e-9691-d452997a4a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:12.594672 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2d426e3c-8522-423e-9691-d452997a4a7b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:12.595894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2d426e3c-8522-423e-9691-d452997a4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:12.691028 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 646/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:10:12.976577 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507538a-f846-45ad-970c-2253623fdc0a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:12.978115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e507538a-f846-45ad-970c-2253623fdc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:13.083279 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507538a-f846-45ad-970c-2253623fdc0a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:13.154193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:13.156445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:13.157282 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:13.157774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:13.158222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:13.159860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:13.265755 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:13.267156 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25702-bd34-4e8e-a9be-b64cdccac2c2 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:13.284737 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 668/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:15.730423 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83eff4-0e13-4f18-9f37-039a7240f8be None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:15.730961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83eff4-0e13-4f18-9f37-039a7240f8be None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:15.730961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83eff4-0e13-4f18-9f37-039a7240f8be None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:15.732839 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 647/1315] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:15.738117 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116c31-8201-469c-92f8-300710f175cf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:15.738533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116c31-8201-469c-92f8-300710f175cf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:15.738696 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116c31-8201-469c-92f8-300710f175cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:15.803968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c9116c31-8201-469c-92f8-300710f175cf None None] MFA Rules not processed for user `ff702c07bb134279b453ea6ea09d38b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:15.805155 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c9116c31-8201-469c-92f8-300710f175cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:15.855573 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:15.899494 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 669/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:16.189236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83eff4-0e13-4f18-9f37-039a7240f8be None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:16.190788 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e83eff4-0e13-4f18-9f37-039a7240f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:16.200395 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:16.290779 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83eff4-0e13-4f18-9f37-039a7240f8be None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:16.362254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:16.362791 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:16.363066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:16.363360 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:16.364907 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:16.425972 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:16.471434 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:16.472856 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ebbbf-1894-4556-9bc8-b936fe271f0b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:16.491326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 648/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:19.187703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0799d26-1f62-4804-8a54-112f1945c87c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:19.188253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0799d26-1f62-4804-8a54-112f1945c87c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:19.188253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0799d26-1f62-4804-8a54-112f1945c87c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:19.189578 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 670/1318] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:19.194887 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32f97-a754-4249-9e0c-15ec025747f9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:19.195254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32f97-a754-4249-9e0c-15ec025747f9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:19.195480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32f97-a754-4249-9e0c-15ec025747f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:19.261387 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-76c32f97-a754-4249-9e0c-15ec025747f9 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:19.262579 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-76c32f97-a754-4249-9e0c-15ec025747f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:19.360742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 649/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:19.649039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0799d26-1f62-4804-8a54-112f1945c87c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:19.650556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e0799d26-1f62-4804-8a54-112f1945c87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:19.753868 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0799d26-1f62-4804-8a54-112f1945c87c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:19.824940 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:19.825542 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:19.825824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:19.826099 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:19.827710 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:19.834411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:19.935405 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:19.936746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c845e-23d9-489e-b7de-a0a4759348a7 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:19.955244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 671/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:22.736358 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aff74-4977-41a4-9bff-64e3c9432668 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:22.736943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aff74-4977-41a4-9bff-64e3c9432668 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:22.736943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aff74-4977-41a4-9bff-64e3c9432668 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:22.738389 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 650/1321] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:22.743920 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc362-a275-41ee-bd64-420df3330ad8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:22.744412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc362-a275-41ee-bd64-420df3330ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:22.744503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc362-a275-41ee-bd64-420df3330ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:22.816824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4bacc362-a275-41ee-bd64-420df3330ad8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:22.818077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4bacc362-a275-41ee-bd64-420df3330ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:22.878895 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:22.922426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 672/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:23.202176 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aff74-4977-41a4-9bff-64e3c9432668 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:23.203797 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-482aff74-4977-41a4-9bff-64e3c9432668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:23.223763 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:23.306044 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aff74-4977-41a4-9bff-64e3c9432668 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:23.378992 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:23.379543 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:23.379824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:23.380097 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:23.381675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:23.452426 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:23.490029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:23.491789 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1809b-42e4-456b-accd-b7045a738a50 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:23.510242 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 651/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:25.737716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b4b3-f015-4522-bc2d-4b981f4a7300 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:25.738268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b4b3-f015-4522-bc2d-4b981f4a7300 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:25.738268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b4b3-f015-4522-bc2d-4b981f4a7300 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:25.739785 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 673/1324] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:25.745100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555969e-40e0-4fc4-b795-0411247984a6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:25.745522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555969e-40e0-4fc4-b795-0411247984a6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:25.745660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555969e-40e0-4fc4-b795-0411247984a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:25.818496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b555969e-40e0-4fc4-b795-0411247984a6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:25.819736 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b555969e-40e0-4fc4-b795-0411247984a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:25.916510 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 652/1325] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:26.210873 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251745eb-1298-4b8d-becd-1ac9aac10d5e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:26.211160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251745eb-1298-4b8d-becd-1ac9aac10d5e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:26.211467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251745eb-1298-4b8d-becd-1ac9aac10d5e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:26.212795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 674/1326] 10.222.0.18 () {54 vars in 963 bytes} [Fri Sep 16 19:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:26.219320 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f095-e634-4d36-9fe6-55e317b0caf1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:26.219647 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f095-e634-4d36-9fe6-55e317b0caf1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:26.219921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8f095-e634-4d36-9fe6-55e317b0caf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:26.251598 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:26.296692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1fe8f095-e634-4d36-9fe6-55e317b0caf1 None None] MFA Rules not processed for user `fc1c6323c0a34d8f9d36fdc95712d26f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:26.298008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1fe8f095-e634-4d36-9fe6-55e317b0caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:26.372855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 653/1327] 10.222.0.18 () {58 vars in 1060 bytes} [Fri Sep 16 19:10:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:26.381587 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251745eb-1298-4b8d-becd-1ac9aac10d5e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:26.383134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-251745eb-1298-4b8d-becd-1ac9aac10d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:26.485459 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251745eb-1298-4b8d-becd-1ac9aac10d5e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:26.562435 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:26.563042 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:26.563344 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:26.563636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:26.565193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:26.582083 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:26.674848 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:26.676205 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebec3ce-70e8-4865-a136-f5c461eb1e20 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:26.693934 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 675/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 19:10:29.131066 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae19bd-2170-414d-ae21-c65a690be589 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:29.131649 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae19bd-2170-414d-ae21-c65a690be589 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:29.131743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae19bd-2170-414d-ae21-c65a690be589 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:29.133105 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 654/1329] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:29.139052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1b41-f2df-465a-a758-24e59938dbc7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:29.139699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1b41-f2df-465a-a758-24e59938dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:29.139699 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1b41-f2df-465a-a758-24e59938dbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:29.209029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a56b1b41-f2df-465a-a758-24e59938dbc7 None None] MFA Rules not processed for user `ff702c07bb134279b453ea6ea09d38b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:29.210250 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a56b1b41-f2df-465a-a758-24e59938dbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:29.277374 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:29.311168 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 676/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:29.595651 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae19bd-2170-414d-ae21-c65a690be589 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:29.597165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-feae19bd-2170-414d-ae21-c65a690be589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:29.708949 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae19bd-2170-414d-ae21-c65a690be589 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:29.731390 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:29.786800 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:29.787810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:29.788091 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:29.788366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:29.789974 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:29.903705 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:29.905078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d8ab4-12c7-497e-94b8-ebaebf977a1b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:29.922810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 655/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:32.317654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b52c2-0100-4e47-a6ac-f7e528581207 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:32.318209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b52c2-0100-4e47-a6ac-f7e528581207 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:32.318209 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b52c2-0100-4e47-a6ac-f7e528581207 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:32.319559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 677/1332] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:32.325292 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fe32-0d49-45e0-a13e-5164ed5ce246 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:32.325696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fe32-0d49-45e0-a13e-5164ed5ce246 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:32.325808 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fe32-0d49-45e0-a13e-5164ed5ce246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:32.387915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:32.390680 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4503fe32-0d49-45e0-a13e-5164ed5ce246 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:32.392495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4503fe32-0d49-45e0-a13e-5164ed5ce246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:32.484721 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 656/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:10:32.765592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b52c2-0100-4e47-a6ac-f7e528581207 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:32.767105 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4f3b52c2-0100-4e47-a6ac-f7e528581207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:32.867692 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b52c2-0100-4e47-a6ac-f7e528581207 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:32.939562 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:32.940159 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:32.940438 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:32.940711 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:32.942267 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:32.968203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:33.050506 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:33.051890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0041c77-d287-4cc9-8192-6d56014e62b6 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:33.071077 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 678/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:35.253549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88556-6f1b-48ec-95a7-e31ad8781b7d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:35.254072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88556-6f1b-48ec-95a7-e31ad8781b7d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:35.254072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88556-6f1b-48ec-95a7-e31ad8781b7d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:35.255455 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 657/1335] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:35.260841 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c719584-76b8-4cb8-9013-f0891c3d3642 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:35.261330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c719584-76b8-4cb8-9013-f0891c3d3642 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:35.261426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c719584-76b8-4cb8-9013-f0891c3d3642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:35.330727 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5c719584-76b8-4cb8-9013-f0891c3d3642 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:35.332389 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5c719584-76b8-4cb8-9013-f0891c3d3642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:35.408983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:35.430792 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 679/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:35.736862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88556-6f1b-48ec-95a7-e31ad8781b7d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:35.738396 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a3a88556-6f1b-48ec-95a7-e31ad8781b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:35.840716 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88556-6f1b-48ec-95a7-e31ad8781b7d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:35.868829 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:35.914213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:35.914814 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:35.915095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:35.915400 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:35.916989 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:36.026537 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:36.028692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517ae79-8d41-4bc5-aea3-7c58da54c33f service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:36.046659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 658/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:38.784718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ccdb3-4b93-4502-b0a2-b07252e945b9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:38.785289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ccdb3-4b93-4502-b0a2-b07252e945b9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:38.785289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ccdb3-4b93-4502-b0a2-b07252e945b9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:38.786588 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 680/1338] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:38.791862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623eceb-7275-482e-9157-987eb80d0501 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:38.792273 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623eceb-7275-482e-9157-987eb80d0501 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:38.792417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623eceb-7275-482e-9157-987eb80d0501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:38.856434 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4623eceb-7275-482e-9157-987eb80d0501 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:38.857632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4623eceb-7275-482e-9157-987eb80d0501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:38.866442 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:38.958623 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 659/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:39.238953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ccdb3-4b93-4502-b0a2-b07252e945b9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:39.240499 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9d3ccdb3-4b93-4502-b0a2-b07252e945b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:39.341572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ccdb3-4b93-4502-b0a2-b07252e945b9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:39.412725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:39.413315 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:39.413592 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:39.413860 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:39.415445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:39.450336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:39.524342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:39.525752 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab73ccc-8a9b-4794-bef9-989599a038c0 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:39.543450 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 681/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:43.801718 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efb98a-f5e1-402c-b6fe-ff318cf59ef8 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:43.802308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efb98a-f5e1-402c-b6fe-ff318cf59ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:43.802308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efb98a-f5e1-402c-b6fe-ff318cf59ef8 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:43.803605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 660/1341] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:43.808855 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4999-2de9-48cf-a969-e4681174bc08 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:43.809275 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4999-2de9-48cf-a969-e4681174bc08 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:43.809383 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b4999-2de9-48cf-a969-e4681174bc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:43.876927 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4b2b4999-2de9-48cf-a969-e4681174bc08 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:43.878142 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4b2b4999-2de9-48cf-a969-e4681174bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:43.963580 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:43.975836 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 682/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:44.261885 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efb98a-f5e1-402c-b6fe-ff318cf59ef8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:44.263457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-73efb98a-f5e1-402c-b6fe-ff318cf59ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:44.368114 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efb98a-f5e1-402c-b6fe-ff318cf59ef8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:44.405852 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:44.442264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:44.442872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:44.443152 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:44.443458 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:44.445040 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:44.554017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:44.555515 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca530854-807e-439d-a9fc-cbb9ba2ad24a service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:44.573672 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 661/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:47.409501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3529a14-93fc-4135-b2ef-868c26a248e8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:47.410060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3529a14-93fc-4135-b2ef-868c26a248e8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:47.410060 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3529a14-93fc-4135-b2ef-868c26a248e8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:47.411454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 683/1344] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:47.416665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399fc82-6482-4a6f-ae90-fe77aea4e943 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:47.417092 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399fc82-6482-4a6f-ae90-fe77aea4e943 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:47.417187 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399fc82-6482-4a6f-ae90-fe77aea4e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:47.481896 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7399fc82-6482-4a6f-ae90-fe77aea4e943 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:47.483104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7399fc82-6482-4a6f-ae90-fe77aea4e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:47.501786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:47.584717 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 662/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:47.875629 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3529a14-93fc-4135-b2ef-868c26a248e8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:47.877177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b3529a14-93fc-4135-b2ef-868c26a248e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:47.982904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3529a14-93fc-4135-b2ef-868c26a248e8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:48.056073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:48.056589 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:48.056864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:48.057134 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:48.059181 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:48.106960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:48.176488 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:48.177953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25062296-b6d9-48b6-a999-b71ad8e3dfb3 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:48.196487 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 684/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:52.290636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29b963-394b-4638-a7e6-d71270515525 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:52.291180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29b963-394b-4638-a7e6-d71270515525 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:52.291180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29b963-394b-4638-a7e6-d71270515525 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:52.292729 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 663/1347] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:10:52.298392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddafb6-de5b-4559-bcba-b33c55c04543 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:52.298859 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddafb6-de5b-4559-bcba-b33c55c04543 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:52.298963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddafb6-de5b-4559-bcba-b33c55c04543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:52.372676 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8fddafb6-de5b-4559-bcba-b33c55c04543 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:52.374078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8fddafb6-de5b-4559-bcba-b33c55c04543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:52.478115 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:52.483337 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 685/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:10:52.767786 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29b963-394b-4638-a7e6-d71270515525 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:10:52.769301 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4d29b963-394b-4638-a7e6-d71270515525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:52.870213 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29b963-394b-4638-a7e6-d71270515525 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:10:52.918095 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:52.943599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:10:52.944195 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:52.944467 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:52.944735 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:52.946296 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:53.054098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:10:53.055488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980585a2-927e-4ce1-9b91-b79cb22b120f service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:10:53.076601 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 664/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:10:57.906092 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7028bd-dd98-44cd-b0b6-4003dd1206d1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:57.906553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7028bd-dd98-44cd-b0b6-4003dd1206d1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:57.906631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7028bd-dd98-44cd-b0b6-4003dd1206d1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:57.908017 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 686/1350] 10.222.0.18 () {54 vars in 921 bytes} [Fri Sep 16 19:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:10:57.913741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b6cc9-6d65-4fb4-8b63-ae8ad486beaa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:10:57.914136 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b6cc9-6d65-4fb4-8b63-ae8ad486beaa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:10:57.914323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b6cc9-6d65-4fb4-8b63-ae8ad486beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:10:58.000015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b88b6cc9-6d65-4fb4-8b63-ae8ad486beaa None None] MFA Rules not processed for user `60a4863b078c436ab224120b8c491f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:10:58.001217 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b88b6cc9-6d65-4fb4-8b63-ae8ad486beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:10:58.023525 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:10:58.160646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 665/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Sep 16 19:10:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:11:02.319013 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775a0c-a238-4e59-91b5-70cb1068fbc7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:02.319610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775a0c-a238-4e59-91b5-70cb1068fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:02.319610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775a0c-a238-4e59-91b5-70cb1068fbc7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:02.321193 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 687/1352] 10.222.0.18 () {54 vars in 921 bytes} [Fri Sep 16 19:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:02.326809 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566b97-cc7b-4c33-9a43-0c92950f3888 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:02.327640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566b97-cc7b-4c33-9a43-0c92950f3888 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:02.328084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566b97-cc7b-4c33-9a43-0c92950f3888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:02.408481 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45566b97-cc7b-4c33-9a43-0c92950f3888 None None] MFA Rules not processed for user `60a4863b078c436ab224120b8c491f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:02.409929 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45566b97-cc7b-4c33-9a43-0c92950f3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:02.485841 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:02.523953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 666/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Sep 16 19:11:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:11:04.005027 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2de65-7276-4224-9d79-2e10599019f8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:04.005559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2de65-7276-4224-9d79-2e10599019f8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:04.005559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2de65-7276-4224-9d79-2e10599019f8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:04.006958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 688/1354] 10.222.0.18 () {54 vars in 921 bytes} [Fri Sep 16 19:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:04.012612 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920feb6-623e-4abd-8255-9912dee9e3d8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:04.013047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920feb6-623e-4abd-8255-9912dee9e3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:04.013175 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920feb6-623e-4abd-8255-9912dee9e3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:04.097015 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7920feb6-623e-4abd-8255-9912dee9e3d8 None None] MFA Rules not processed for user `60a4863b078c436ab224120b8c491f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:04.098272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7920feb6-623e-4abd-8255-9912dee9e3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:04.208051 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 667/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Fri Sep 16 19:11:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:11:07.750718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2e07f-d7df-45ab-a7d5-399944ddbb70 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:07.753689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2e07f-d7df-45ab-a7d5-399944ddbb70 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:07.754041 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2e07f-d7df-45ab-a7d5-399944ddbb70 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:07.757639 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 689/1356] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:07.764834 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92d08-a3a4-4dc7-9d94-bfa463bf97c2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:07.766496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92d08-a3a4-4dc7-9d94-bfa463bf97c2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:07.766844 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92d08-a3a4-4dc7-9d94-bfa463bf97c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:07.827166 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:07.861201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5ae92d08-a3a4-4dc7-9d94-bfa463bf97c2 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:07.862521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5ae92d08-a3a4-4dc7-9d94-bfa463bf97c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:07.975640 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 668/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:08.272639 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c32e6-4c46-4286-a726-223a8422226b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:08.273677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c32e6-4c46-4286-a726-223a8422226b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:08.274563 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c32e6-4c46-4286-a726-223a8422226b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:08.278138 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 690/1358] 10.222.0.18 () {54 vars in 965 bytes} [Fri Sep 16 19:11:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:08.288032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ee7e-18b7-4bc9-aa89-897eb0213ee3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:08.289062 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ee7e-18b7-4bc9-aa89-897eb0213ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:08.289527 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ee7e-18b7-4bc9-aa89-897eb0213ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:08.371272 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4520ee7e-18b7-4bc9-aa89-897eb0213ee3 None None] MFA Rules not processed for user `8d7573ee32f6443892a361f012c2f204`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:08.372494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4520ee7e-18b7-4bc9-aa89-897eb0213ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:08.424029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:08.495766 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 669/1359] 10.222.0.18 () {58 vars in 1062 bytes} [Fri Sep 16 19:11:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:08.503479 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c32e6-4c46-4286-a726-223a8422226b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:08.505078 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c0c32e6-4c46-4286-a726-223a8422226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:08.662034 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c32e6-4c46-4286-a726-223a8422226b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:08.741423 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:08.785414 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af304325-7540-48cd-8122-942756c7246a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:08.786147 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304325-7540-48cd-8122-942756c7246a service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:08.786524 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304325-7540-48cd-8122-942756c7246a service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:08.786889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af304325-7540-48cd-8122-942756c7246a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:08.788623 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-af304325-7540-48cd-8122-942756c7246a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:08.887652 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af304325-7540-48cd-8122-942756c7246a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:08.890622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af304325-7540-48cd-8122-942756c7246a service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:08.910342 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 691/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:11.315578 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092baaf7-3a5e-458e-ac61-7fabbe4fcdd0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:11.316029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092baaf7-3a5e-458e-ac61-7fabbe4fcdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:11.316029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092baaf7-3a5e-458e-ac61-7fabbe4fcdd0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:11.317915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 670/1361] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:11.329166 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc888-520e-47c2-a32d-256e754a0d0f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:11.329671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc888-520e-47c2-a32d-256e754a0d0f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:11.329671 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc888-520e-47c2-a32d-256e754a0d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:11.398150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-742dc888-520e-47c2-a32d-256e754a0d0f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:11.399359 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-742dc888-520e-47c2-a32d-256e754a0d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:11.442145 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:11.503982 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 692/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:11.796894 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092baaf7-3a5e-458e-ac61-7fabbe4fcdd0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:11.798422 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-092baaf7-3a5e-458e-ac61-7fabbe4fcdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:11.973374 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092baaf7-3a5e-458e-ac61-7fabbe4fcdd0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:11.985854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:12.116981 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:12.117595 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:12.117873 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:12.118144 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:12.121772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:12.216893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:12.218509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de70ac-177d-4338-b1a1-a14c0ccd6b52 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:12.237309 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 671/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:14.735950 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222ea01-69dc-4ca0-82c3-2c8ab48abd52 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:14.736486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222ea01-69dc-4ca0-82c3-2c8ab48abd52 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:14.736486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222ea01-69dc-4ca0-82c3-2c8ab48abd52 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:14.737762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 693/1364] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:14.743495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7dad5-c99b-4b29-a52a-78653f5806ab None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:14.743933 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7dad5-c99b-4b29-a52a-78653f5806ab None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:14.744014 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7dad5-c99b-4b29-a52a-78653f5806ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:14.761416 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:14.814062 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-caf7dad5-c99b-4b29-a52a-78653f5806ab None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:14.815254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-caf7dad5-c99b-4b29-a52a-78653f5806ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:14.932365 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 672/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:15.225842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4d1e-2075-4377-9246-f7f5a75f02cb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:15.226247 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4d1e-2075-4377-9246-f7f5a75f02cb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:15.226634 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4d1e-2075-4377-9246-f7f5a75f02cb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:15.228588 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 694/1366] 10.222.0.18 () {54 vars in 965 bytes} [Fri Sep 16 19:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:15.234002 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f2fb-327c-4bdb-af33-673716c463a5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:15.234295 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f2fb-327c-4bdb-af33-673716c463a5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:15.234570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9f2fb-327c-4bdb-af33-673716c463a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:15.311983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-81f9f2fb-327c-4bdb-af33-673716c463a5 None None] MFA Rules not processed for user `8d7573ee32f6443892a361f012c2f204`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:15.313180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-81f9f2fb-327c-4bdb-af33-673716c463a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:15.349062 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:15.434155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 673/1367] 10.222.0.18 () {58 vars in 1062 bytes} [Fri Sep 16 19:11:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:15.441968 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4d1e-2075-4377-9246-f7f5a75f02cb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:15.443484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a1ed4d1e-2075-4377-9246-f7f5a75f02cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:15.597607 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4d1e-2075-4377-9246-f7f5a75f02cb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:15.618407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:15.716552 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:15.717151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:15.717426 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:15.717695 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:15.719241 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:15.815116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:15.816730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd6b91-0f7f-40ed-954b-3cdf0954e0df service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:15.836655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 695/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:15.965549 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f039-2c10-4ef2-bf7b-e00de8331348 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:15.966113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f039-2c10-4ef2-bf7b-e00de8331348 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:15.966113 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f039-2c10-4ef2-bf7b-e00de8331348 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:15.967719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 674/1369] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:15.973703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d244ae-7e1d-446c-8a6b-7746da759324 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:15.974066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d244ae-7e1d-446c-8a6b-7746da759324 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:15.974263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d244ae-7e1d-446c-8a6b-7746da759324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:16.011650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:16.056500 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-77d244ae-7e1d-446c-8a6b-7746da759324 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:16.057795 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-77d244ae-7e1d-446c-8a6b-7746da759324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:16.185236 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 696/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:16.491147 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c6363-d128-4f95-97a3-ce641988ea04 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:16.491593 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c6363-d128-4f95-97a3-ce641988ea04 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:16.491980 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c6363-d128-4f95-97a3-ce641988ea04 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:16.493570 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 675/1371] 10.222.0.18 () {54 vars in 964 bytes} [Fri Sep 16 19:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:16.499091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ee45f-5a96-470e-a6a6-bdbaa271f7a9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:16.499541 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ee45f-5a96-470e-a6a6-bdbaa271f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:16.499924 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ee45f-5a96-470e-a6a6-bdbaa271f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:16.591679 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e24ee45f-5a96-470e-a6a6-bdbaa271f7a9 None None] MFA Rules not processed for user `0b094a1d708f4b02b83405e6befbed9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:16.593028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e24ee45f-5a96-470e-a6a6-bdbaa271f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:16.644521 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:16.678617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 697/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Fri Sep 16 19:11:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:16.686306 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c6363-d128-4f95-97a3-ce641988ea04 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:16.688084 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-702c6363-d128-4f95-97a3-ce641988ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:16.805457 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c6363-d128-4f95-97a3-ce641988ea04 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:16.855433 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:16.919696 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b094a1d708f4b02b83405e6befbed9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:16.920660 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:16.921121 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:16.921654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:16.924165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:17.089165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:17.089165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b782002-9021-406c-94c8-0a1712fa1969 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:17.117715 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 676/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:18.119234 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f297-23a9-4b14-a927-67168641f8d9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:18.120281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f297-23a9-4b14-a927-67168641f8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:18.120371 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f297-23a9-4b14-a927-67168641f8d9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:18.121702 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 698/1374] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:18.127067 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2b62-b395-4373-a6dd-3e81e652aa13 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:18.127521 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2b62-b395-4373-a6dd-3e81e652aa13 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:18.127654 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2b62-b395-4373-a6dd-3e81e652aa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:18.194723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5d2d2b62-b395-4373-a6dd-3e81e652aa13 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:18.195968 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5d2d2b62-b395-4373-a6dd-3e81e652aa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:18.217078 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:18.293209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 677/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:18.580033 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f297-23a9-4b14-a927-67168641f8d9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:18.581545 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-be42f297-23a9-4b14-a927-67168641f8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:18.749452 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f297-23a9-4b14-a927-67168641f8d9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:18.807615 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:18.870439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:18.871054 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:18.871360 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:18.871635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:18.873255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:18.964657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:18.966235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114735f3-a394-460b-9f07-1df34da7cf40 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:18.988974 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 699/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:21.213117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8ffbf-2057-4671-bc44-158c7e923fba None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:21.213752 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8ffbf-2057-4671-bc44-158c7e923fba None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:21.213752 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8ffbf-2057-4671-bc44-158c7e923fba None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:21.215082 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 678/1377] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:21.220388 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9d39b-5ea3-44e4-81a8-2aaaef5269d8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:21.220847 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9d39b-5ea3-44e4-81a8-2aaaef5269d8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:21.220929 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9d39b-5ea3-44e4-81a8-2aaaef5269d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:21.302339 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2bd9d39b-5ea3-44e4-81a8-2aaaef5269d8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:21.303993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2bd9d39b-5ea3-44e4-81a8-2aaaef5269d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:21.312954 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:21.415343 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 700/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:21.702168 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8ffbf-2057-4671-bc44-158c7e923fba None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:21.703862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-46f8ffbf-2057-4671-bc44-158c7e923fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:21.866821 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8ffbf-2057-4671-bc44-158c7e923fba None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:21.871390 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:21.988037 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:21.988552 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:21.988827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:21.989093 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:21.990643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:22.088862 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:22.090461 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468a23e-786d-478e-917d-275fe64b2bb5 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:22.108824 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 679/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:24.328120 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3681e-4f21-4f55-9d84-602fd456077a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:24.328656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3681e-4f21-4f55-9d84-602fd456077a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:24.328656 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3681e-4f21-4f55-9d84-602fd456077a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:24.330079 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 701/1380] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:24.335076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62049f-5652-4d70-acc1-5411cd5aa116 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:24.335496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62049f-5652-4d70-acc1-5411cd5aa116 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:24.335719 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62049f-5652-4d70-acc1-5411cd5aa116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:24.348215 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:24.407768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0e62049f-5652-4d70-acc1-5411cd5aa116 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:24.409032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0e62049f-5652-4d70-acc1-5411cd5aa116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:24.527930 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 680/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:24.809028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3681e-4f21-4f55-9d84-602fd456077a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:24.810584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cea3681e-4f21-4f55-9d84-602fd456077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:24.961667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:24.974091 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3681e-4f21-4f55-9d84-602fd456077a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:25.096553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:25.097160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:25.097439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:25.097718 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:25.099340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:25.221508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:25.223750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a0b30-74e0-42d4-9332-33bd7644b98e service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:25.237892 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:25.247657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 702/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:27.437776 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a59db3-1e9a-4d5f-b432-ea4ba89aed31 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:27.438303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a59db3-1e9a-4d5f-b432-ea4ba89aed31 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:27.438509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a59db3-1e9a-4d5f-b432-ea4ba89aed31 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:27.440522 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 681/1383] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:27.445801 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aae283-74d8-4363-b4a7-6073dc636355 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:27.446242 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aae283-74d8-4363-b4a7-6073dc636355 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:27.446346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aae283-74d8-4363-b4a7-6073dc636355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:27.513716 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-38aae283-74d8-4363-b4a7-6073dc636355 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:27.514928 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-38aae283-74d8-4363-b4a7-6073dc636355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:27.628330 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 703/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:28.025432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a59db3-1e9a-4d5f-b432-ea4ba89aed31 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:28.028238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c3a59db3-1e9a-4d5f-b432-ea4ba89aed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:28.122155 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:28.186185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a59db3-1e9a-4d5f-b432-ea4ba89aed31 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:28.302173 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:28.303004 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:28.303488 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:28.303854 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:28.306207 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:28.376342 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:28.415897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:28.417983 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b1cbb-cacb-4c7d-8b77-53a586647e0b service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:28.436009 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 682/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:31.221158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5bc5-f64d-40de-a218-81e7e58b2246 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:31.221746 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5bc5-f64d-40de-a218-81e7e58b2246 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:31.221929 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5bc5-f64d-40de-a218-81e7e58b2246 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:31.223981 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 704/1386] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:31.229815 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630c155-b76f-4e0f-93cd-752575b9e5f8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:31.230229 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630c155-b76f-4e0f-93cd-752575b9e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:31.230354 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630c155-b76f-4e0f-93cd-752575b9e5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:31.297380 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a630c155-b76f-4e0f-93cd-752575b9e5f8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:31.298589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a630c155-b76f-4e0f-93cd-752575b9e5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:31.397872 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 683/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:31.689990 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db5bc5-f64d-40de-a218-81e7e58b2246 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:31.691527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-94db5bc5-f64d-40de-a218-81e7e58b2246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:31.768161 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:31.843094 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db5bc5-f64d-40de-a218-81e7e58b2246 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:31.960203 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:31.960757 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:31.961031 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:31.961299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:31.962893 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:32.002262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:32.053525 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:32.055105 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe47e6f-c831-44cb-bc28-b3b945f70f4a service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:32.075331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 705/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:33.361369 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a206fd-7da8-4c56-b89a-d75e994b28b7 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:33.361876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a206fd-7da8-4c56-b89a-d75e994b28b7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:33.361876 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a206fd-7da8-4c56-b89a-d75e994b28b7 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:33.363196 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 684/1389] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:33.368781 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07993cff-83ee-4d6c-921d-4038284ae027 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:33.369748 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07993cff-83ee-4d6c-921d-4038284ae027 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:33.369877 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07993cff-83ee-4d6c-921d-4038284ae027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:33.435967 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-07993cff-83ee-4d6c-921d-4038284ae027 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:33.437158 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07993cff-83ee-4d6c-921d-4038284ae027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:33.531213 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 706/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:33.828582 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa577918-f882-4af0-a8c3-b1175caa2328 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:33.828939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa577918-f882-4af0-a8c3-b1175caa2328 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:33.829411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa577918-f882-4af0-a8c3-b1175caa2328 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:33.830840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 685/1391] 10.222.0.18 () {54 vars in 962 bytes} [Fri Sep 16 19:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:33.836195 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea7b8-745a-4d28-a5b1-bc9974e97568 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:33.836549 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea7b8-745a-4d28-a5b1-bc9974e97568 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:33.836890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea7b8-745a-4d28-a5b1-bc9974e97568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:33.852429 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:33.914378 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8c8ea7b8-745a-4d28-a5b1-bc9974e97568 None None] MFA Rules not processed for user `e7f6b6c5a3f0462d8da8b4a7afec5335`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:33.915606 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8c8ea7b8-745a-4d28-a5b1-bc9974e97568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:33.991509 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 707/1392] 10.222.0.18 () {58 vars in 1059 bytes} [Fri Sep 16 19:11:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:34.000984 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa577918-f882-4af0-a8c3-b1175caa2328 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:34.002632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aa577918-f882-4af0-a8c3-b1175caa2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:34.034209 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:34.108276 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa577918-f882-4af0-a8c3-b1175caa2328 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:34.208681 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f6b6c5a3f0462d8da8b4a7afec5335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:34.209303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:34.209563 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:34.209826 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:34.211415 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:34.277200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d22d1-c62f-4dfc-8a98-73482821f2cf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:34.277497 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d22d1-c62f-4dfc-8a98-73482821f2cf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:34.277776 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d22d1-c62f-4dfc-8a98-73482821f2cf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:34.279109 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 708/1393] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 19:11:34.284303 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc41cd1-a06d-4afe-b445-8cbb5db0b116 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:34.284584 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc41cd1-a06d-4afe-b445-8cbb5db0b116 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:34.284856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc41cd1-a06d-4afe-b445-8cbb5db0b116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:34.293487 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:34.327567 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:34.329120 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:34.347658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 686/1394] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:34.353221 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bcc41cd1-a06d-4afe-b445-8cbb5db0b116 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:34.354906 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bcc41cd1-a06d-4afe-b445-8cbb5db0b116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:34.409535 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:34.451503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 709/1395] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 19:11:34.743496 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:34.744792 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:34.849273 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940e816-784d-433b-823c-06cfcdb5cfb9 service glance] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:34.934062 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:34.972524 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:34.973146 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:34.973425 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:34.973695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:34.975417 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:35.077408 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:35.079742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cce37f-b66a-4b21-90ce-6940958f9b92 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:35.101838 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 687/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:35.137434 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da6450-167b-4cb7-ad81-85bf5ffebae6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:35.137825 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da6450-167b-4cb7-ad81-85bf5ffebae6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:35.138163 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da6450-167b-4cb7-ad81-85bf5ffebae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:35.213582 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d8da6450-167b-4cb7-ad81-85bf5ffebae6 None None] MFA Rules not processed for user `355f8f0689c54248b22dbeb1b578c77e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:35.214861 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d8da6450-167b-4cb7-ad81-85bf5ffebae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:35.309749 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 710/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Fri Sep 16 19:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:35.354123 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cb721-ba57-4c80-a061-da1a7d1463af None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:35.354482 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cb721-ba57-4c80-a061-da1a7d1463af None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:35.354827 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cb721-ba57-4c80-a061-da1a7d1463af None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:35.356314 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 688/1398] 10.222.0.18 () {54 vars in 943 bytes} [Fri Sep 16 19:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:35.363200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05e0b1-ca6e-4ea2-be80-fdb7da560190 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:35.363533 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05e0b1-ca6e-4ea2-be80-fdb7da560190 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:35.363808 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05e0b1-ca6e-4ea2-be80-fdb7da560190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:35.422146 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:35.440484 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2e05e0b1-ca6e-4ea2-be80-fdb7da560190 None None] MFA Rules not processed for user `cb269fa787664ebead0e2e9fd21ce219`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:35.441689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2e05e0b1-ca6e-4ea2-be80-fdb7da560190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:35.546856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 711/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Fri Sep 16 19:11:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:35.556334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cb721-ba57-4c80-a061-da1a7d1463af None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:35.557830 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-036cb721-ba57-4c80-a061-da1a7d1463af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:35.653298 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:35.700326 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cb721-ba57-4c80-a061-da1a7d1463af None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:35.804954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb269fa787664ebead0e2e9fd21ce219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:35.805602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:35.805828 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:35.806102 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:35.807795 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:35.885149 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:35.894658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:35.896312 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a4b29-4d7c-42e0-972e-888059b999e7 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:35.904919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 689/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Fri Sep 16 19:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 19:11:37.327222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1161-ab45-438d-94cd-cf23767d996a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:37.327909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1161-ab45-438d-94cd-cf23767d996a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:37.327909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1161-ab45-438d-94cd-cf23767d996a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:37.329048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 712/1401] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:37.334266 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968f5b-c7a7-4f59-b3ca-1f00cbc35720 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:37.334704 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968f5b-c7a7-4f59-b3ca-1f00cbc35720 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:37.334790 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968f5b-c7a7-4f59-b3ca-1f00cbc35720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:37.407758 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-41968f5b-c7a7-4f59-b3ca-1f00cbc35720 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:37.409269 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-41968f5b-c7a7-4f59-b3ca-1f00cbc35720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:37.515490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 690/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:37.810310 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1161-ab45-438d-94cd-cf23767d996a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:37.812029 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-804e1161-ab45-438d-94cd-cf23767d996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:37.934135 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:37.984295 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1161-ab45-438d-94cd-cf23767d996a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:38.130189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:38.130838 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:38.131114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:38.131409 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:38.133028 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:38.225955 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:38.249631 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:38.251449 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665d98a-f4bc-4f95-af2e-6b1d8d0dc957 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:38.270553 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 713/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:40.549438 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0343500-be18-493f-9408-57d38802fc44 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:40.549959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0343500-be18-493f-9408-57d38802fc44 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:40.549959 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0343500-be18-493f-9408-57d38802fc44 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:40.551507 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 691/1404] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:40.557487 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4541-e74e-4270-a98b-f76d3dd41de9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:40.557949 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4541-e74e-4270-a98b-f76d3dd41de9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:40.558048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4541-e74e-4270-a98b-f76d3dd41de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:40.634581 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-af8d4541-e74e-4270-a98b-f76d3dd41de9 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:40.635816 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-af8d4541-e74e-4270-a98b-f76d3dd41de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:40.746738 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 714/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:41.044008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0343500-be18-493f-9408-57d38802fc44 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:41.045530 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f0343500-be18-493f-9408-57d38802fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:41.140972 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:41.226285 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0343500-be18-493f-9408-57d38802fc44 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:41.361871 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:41.362807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:41.363180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:41.363484 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:41.366506 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:41.417384 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:41.481123 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:41.482708 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562139b-b4fa-4d31-9dbc-91377e594bfa service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:41.503420 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 692/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:43.701149 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d17b6-07fe-4235-a0dd-2ad6e12b05b5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:43.701706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d17b6-07fe-4235-a0dd-2ad6e12b05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:43.701706 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d17b6-07fe-4235-a0dd-2ad6e12b05b5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:43.703669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 715/1407] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:43.710491 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18f001-d1be-46c1-a12f-355993b37d5d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:43.711659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18f001-d1be-46c1-a12f-355993b37d5d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:43.712761 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18f001-d1be-46c1-a12f-355993b37d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:43.790886 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bc18f001-d1be-46c1-a12f-355993b37d5d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:43.792171 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc18f001-d1be-46c1-a12f-355993b37d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:43.900643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 693/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:44.192685 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d17b6-07fe-4235-a0dd-2ad6e12b05b5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:11:44.194189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-343d17b6-07fe-4235-a0dd-2ad6e12b05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:44.258563 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:44.367074 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d17b6-07fe-4235-a0dd-2ad6e12b05b5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:11:44.484185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:11:44.484791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:44.485071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:44.485346 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:44.486965 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:44.497163 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:44.593148 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:11:44.594762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181d14e-425f-4995-9421-3dafa7ea453b service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:11:44.615703 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 716/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:11:53.898333 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:54.610672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=41 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:58.674918 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e63051-2878-44a7-9a01-6977464ad869 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:58.675576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e63051-2878-44a7-9a01-6977464ad869 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:58.675576 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e63051-2878-44a7-9a01-6977464ad869 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:58.676878 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 694/1410] 10.222.0.18 () {54 vars in 918 bytes} [Fri Sep 16 19:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:58.681837 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b256067-54ec-40c8-bb25-b982bcadd73e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:58.682232 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b256067-54ec-40c8-bb25-b982bcadd73e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:58.682370 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b256067-54ec-40c8-bb25-b982bcadd73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:58.758172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6b256067-54ec-40c8-bb25-b982bcadd73e None None] MFA Rules not processed for user `60a4863b078c436ab224120b8c491f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:58.759401 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6b256067-54ec-40c8-bb25-b982bcadd73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:58.861963 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 717/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Fri Sep 16 19:11:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:11:58.876212 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d08c4-16b8-4f56-b467-66cc59eecabe None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:58.876494 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d08c4-16b8-4f56-b467-66cc59eecabe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:58.876768 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d08c4-16b8-4f56-b467-66cc59eecabe None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:58.878088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 695/1412] 10.222.0.18 () {54 vars in 918 bytes} [Fri Sep 16 19:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:11:58.883154 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc230e78-3dda-4ea1-9e0f-42d20d13b3d1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:11:58.883471 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc230e78-3dda-4ea1-9e0f-42d20d13b3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:11:58.883750 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc230e78-3dda-4ea1-9e0f-42d20d13b3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:11:58.955132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:11:58.958318 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bc230e78-3dda-4ea1-9e0f-42d20d13b3d1 None None] MFA Rules not processed for user `8d7573ee32f6443892a361f012c2f204`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:11:58.960052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc230e78-3dda-4ea1-9e0f-42d20d13b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:11:59.071828 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 718/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Fri Sep 16 19:11:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:01.659495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33af5-1832-4a98-9bfc-251817353fd6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:01.660060 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33af5-1832-4a98-9bfc-251817353fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:01.660060 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33af5-1832-4a98-9bfc-251817353fd6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:01.661439 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:01.667249 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec76d5-684a-4efd-b19c-fc734284b891 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:01.667675 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec76d5-684a-4efd-b19c-fc734284b891 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:01.667827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec76d5-684a-4efd-b19c-fc734284b891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:01.732596 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-edec76d5-684a-4efd-b19c-fc734284b891 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:01.733787 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-edec76d5-684a-4efd-b19c-fc734284b891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:01.787071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:01.828048 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:02.117999 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33af5-1832-4a98-9bfc-251817353fd6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:02.119767 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2db33af5-1832-4a98-9bfc-251817353fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:02.264509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33af5-1832-4a98-9bfc-251817353fd6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:02.336641 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:02.377154 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:02.377770 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:02.378050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:02.378324 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:02.379933 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:02.470842 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:02.472473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899ea3c-b3a2-4a1c-bdfe-4bef0e792ef6 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:02.490252 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 697/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:03.654586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370c8e-a4a6-455e-bf22-36ffdf89635d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:03.655130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370c8e-a4a6-455e-bf22-36ffdf89635d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:03.655130 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370c8e-a4a6-455e-bf22-36ffdf89635d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:03.656507 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:03.662725 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0524d-a7dc-4200-8c7f-61ba98d7af28 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:03.663181 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0524d-a7dc-4200-8c7f-61ba98d7af28 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:03.663323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0524d-a7dc-4200-8c7f-61ba98d7af28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:03.742695 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-50c0524d-a7dc-4200-8c7f-61ba98d7af28 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:03.743943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-50c0524d-a7dc-4200-8c7f-61ba98d7af28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:03.769972 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:03.848023 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:03.998246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16370c8e-a4a6-455e-bf22-36ffdf89635d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:03.999791 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-16370c8e-a4a6-455e-bf22-36ffdf89635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:04.144194 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16370c8e-a4a6-455e-bf22-36ffdf89635d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:04.191894 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:04.265080 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:04.265684 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:04.265959 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:04.266225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:04.267821 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:04.354655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:04.356249 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f1f38-ce39-478e-a45b-87fa07f937c1 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:04.373783 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 721/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:05.511245 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1409d6-6189-4c3f-9fed-d3f42ae0230d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:05.511783 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1409d6-6189-4c3f-9fed-d3f42ae0230d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:05.511901 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1409d6-6189-4c3f-9fed-d3f42ae0230d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:05.513172 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:05.519489 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f62f1e-3a25-433f-b3b8-0929025ead78 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:05.519900 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f62f1e-3a25-433f-b3b8-0929025ead78 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:05.520039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f62f1e-3a25-433f-b3b8-0929025ead78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:05.577244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:05.599953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-07f62f1e-3a25-433f-b3b8-0929025ead78 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:05.601164 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07f62f1e-3a25-433f-b3b8-0929025ead78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:05.699669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:05.851264 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1409d6-6189-4c3f-9fed-d3f42ae0230d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:05.852701 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6a1409d6-6189-4c3f-9fed-d3f42ae0230d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:05.995376 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1409d6-6189-4c3f-9fed-d3f42ae0230d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:06.007254 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:06.108575 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:06.109116 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:06.109393 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:06.109663 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:06.111308 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:06.199364 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:06.200939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4557a7-b8a3-4ec1-ab8a-df0065372b4e service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:06.218979 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 700/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:07.313143 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2115a-fd69-4489-9df4-ed42494f676b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:07.313610 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2115a-fd69-4489-9df4-ed42494f676b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:07.313736 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2115a-fd69-4489-9df4-ed42494f676b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:07.315012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:07.321676 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279d409-8b26-4c7a-992c-2ae8e34744f6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:07.322135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279d409-8b26-4c7a-992c-2ae8e34744f6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:07.322228 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279d409-8b26-4c7a-992c-2ae8e34744f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:07.342283 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:07.397517 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3279d409-8b26-4c7a-992c-2ae8e34744f6 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:07.399392 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3279d409-8b26-4c7a-992c-2ae8e34744f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:07.492411 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:07.643412 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2115a-fd69-4489-9df4-ed42494f676b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:07.644854 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d1a2115a-fd69-4489-9df4-ed42494f676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:07.755064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:07.795774 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2115a-fd69-4489-9df4-ed42494f676b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:07.908096 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:07.908641 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:07.908917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:07.909188 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:07.910761 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:07.985173 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:07.999501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:08.001084 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722eb11-9c4d-402b-a58b-e0854c745682 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:08.018554 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 724/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:09.126519 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc746d7d-3dfc-4411-83d0-2f42f9785e12 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:09.127017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc746d7d-3dfc-4411-83d0-2f42f9785e12 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:09.127141 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc746d7d-3dfc-4411-83d0-2f42f9785e12 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:09.128456 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:09.135570 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99178dc-3ffd-4f3d-baea-e366b484dcc8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:09.136064 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99178dc-3ffd-4f3d-baea-e366b484dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:09.136160 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99178dc-3ffd-4f3d-baea-e366b484dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:09.209603 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b99178dc-3ffd-4f3d-baea-e366b484dcc8 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:09.210862 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b99178dc-3ffd-4f3d-baea-e366b484dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:09.307257 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:09.462117 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc746d7d-3dfc-4411-83d0-2f42f9785e12 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:09.463602 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fc746d7d-3dfc-4411-83d0-2f42f9785e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:09.545727 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:09.610867 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc746d7d-3dfc-4411-83d0-2f42f9785e12 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:09.721919 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:09.722436 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:09.722709 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:09.722977 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:09.724569 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:09.771799 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:09.812129 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:09.813728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41c6ac-576a-4ea0-a5dd-01652cc5d41b service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:09.832067 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 703/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:10.883677 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4360c-875a-4b76-b8d7-6db5afea4fce None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:10.884309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4360c-875a-4b76-b8d7-6db5afea4fce None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:10.884309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4360c-875a-4b76-b8d7-6db5afea4fce None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:10.885565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:10.891694 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b6193-d1ee-46f0-80a4-cced36991a01 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:10.892151 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b6193-d1ee-46f0-80a4-cced36991a01 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:10.892334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b6193-d1ee-46f0-80a4-cced36991a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:10.966220 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e06b6193-d1ee-46f0-80a4-cced36991a01 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:10.967611 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e06b6193-d1ee-46f0-80a4-cced36991a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:11.061677 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:11.212654 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4360c-875a-4b76-b8d7-6db5afea4fce None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:11.214210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3fc4360c-875a-4b76-b8d7-6db5afea4fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:11.262614 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:11.357014 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4360c-875a-4b76-b8d7-6db5afea4fce None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:11.466039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:11.466636 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:11.466912 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:11.467184 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:11.468784 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:11.475624 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:11.558022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:11.559668 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59960c-3a87-4729-8836-37fdb18f8d22 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:11.577919 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 727/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 19:12:12.625621 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2a01-4d6e-4451-8aaf-275b4ec40438 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:12.626185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2a01-4d6e-4451-8aaf-275b4ec40438 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:12.626185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2a01-4d6e-4451-8aaf-275b4ec40438 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:12.627533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:12.634144 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912970f-da03-4b29-abf7-b826f255b44b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:12.634586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912970f-da03-4b29-abf7-b826f255b44b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:12.634689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912970f-da03-4b29-abf7-b826f255b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:12.709116 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8912970f-da03-4b29-abf7-b826f255b44b None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:12.710898 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8912970f-da03-4b29-abf7-b826f255b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:12.779053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:12.807849 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:12.962636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2a01-4d6e-4451-8aaf-275b4ec40438 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:12.964101 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e1f2a01-4d6e-4451-8aaf-275b4ec40438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:12.981377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:13.107303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2a01-4d6e-4451-8aaf-275b4ec40438 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:13.194334 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:13.219883 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8494-3172-452c-b061-41774156daf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:13.220428 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8494-3172-452c-b061-41774156daf2 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:13.220710 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8494-3172-452c-b061-41774156daf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:13.220982 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8494-3172-452c-b061-41774156daf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:13.222566 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-689d8494-3172-452c-b061-41774156daf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:13.310811 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d8494-3172-452c-b061-41774156daf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:13.312447 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d8494-3172-452c-b061-41774156daf2 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:13.329775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 706/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:14.373298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364e4a-674f-4e95-9470-deb6ce4e7fc1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:14.373811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364e4a-674f-4e95-9470-deb6ce4e7fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:14.373811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364e4a-674f-4e95-9470-deb6ce4e7fc1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:14.375089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:14.381075 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d087fd-c183-4193-be80-c3f86943d8b4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:14.381533 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d087fd-c183-4193-be80-c3f86943d8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:14.381735 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d087fd-c183-4193-be80-c3f86943d8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:14.456029 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-67d087fd-c183-4193-be80-c3f86943d8b4 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:14.457249 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67d087fd-c183-4193-be80-c3f86943d8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:14.498617 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:14.552352 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:14.701822 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364e4a-674f-4e95-9470-deb6ce4e7fc1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:14.703244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-28364e4a-674f-4e95-9470-deb6ce4e7fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:14.850235 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364e4a-674f-4e95-9470-deb6ce4e7fc1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:14.903090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:14.961979 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:14.962513 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:14.962790 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:14.963059 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:14.964653 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:15.051531 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:15.053114 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218abc2b-c10e-4a35-bca0-f147b2579d79 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:15.071891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 730/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:16.141323 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116c9ba-213f-48e8-a863-a9db67e76b93 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:16.141846 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116c9ba-213f-48e8-a863-a9db67e76b93 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:16.141953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116c9ba-213f-48e8-a863-a9db67e76b93 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:16.143224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 19:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:16.149466 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30831ec-17a1-4dd1-bfb4-6775f7b451b9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:16.149890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30831ec-17a1-4dd1-bfb4-6775f7b451b9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:16.150004 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30831ec-17a1-4dd1-bfb4-6775f7b451b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:16.234736 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f30831ec-17a1-4dd1-bfb4-6775f7b451b9 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:16.236018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f30831ec-17a1-4dd1-bfb4-6775f7b451b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:16.238039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:16.331298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 19:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:16.481746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116c9ba-213f-48e8-a863-a9db67e76b93 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:16.483262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9116c9ba-213f-48e8-a863-a9db67e76b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:16.635728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116c9ba-213f-48e8-a863-a9db67e76b93 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:16.664135 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:16.749072 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:16.749678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:16.749960 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:16.750232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:16.751853 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:16.839108 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:16.840745 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3af37-1021-4fa3-ab26-5afc06dbd228 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:16.859031 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 709/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:18.933446 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048287b6-094f-4c05-a813-7360965f060a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:18.934037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048287b6-094f-4c05-a813-7360965f060a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:18.934037 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048287b6-094f-4c05-a813-7360965f060a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:18.935336 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:18.940678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e20a-4c66-4c26-bc32-3074a99e7d17 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:18.941104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e20a-4c66-4c26-bc32-3074a99e7d17 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:18.941224 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e20a-4c66-4c26-bc32-3074a99e7d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:18.981826 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:19.006589 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7464e20a-4c66-4c26-bc32-3074a99e7d17 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:19.007807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7464e20a-4c66-4c26-bc32-3074a99e7d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:19.102962 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:19.391072 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048287b6-094f-4c05-a813-7360965f060a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:19.392617 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-048287b6-094f-4c05-a813-7360965f060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:19.498692 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048287b6-094f-4c05-a813-7360965f060a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:19.532842 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:19.574307 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:19.574918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:19.575189 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:19.575482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:19.577056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:19.686222 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:19.687586 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb6634-fb17-44f2-8f0a-38f24e8dcf1c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:19.705377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 733/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:22.083377 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a637f0-b2f3-42dd-98e3-2f5bfbf40356 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:22.083788 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a637f0-b2f3-42dd-98e3-2f5bfbf40356 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:22.083941 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a637f0-b2f3-42dd-98e3-2f5bfbf40356 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:22.085861 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:22.091274 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78869e04-31c9-4cbd-b2bd-2556000c91b4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:22.091732 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78869e04-31c9-4cbd-b2bd-2556000c91b4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:22.091827 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78869e04-31c9-4cbd-b2bd-2556000c91b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:22.157043 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-78869e04-31c9-4cbd-b2bd-2556000c91b4 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:22.158219 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-78869e04-31c9-4cbd-b2bd-2556000c91b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:22.175926 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:22.260118 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:22.546096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a637f0-b2f3-42dd-98e3-2f5bfbf40356 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:22.547665 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d2a637f0-b2f3-42dd-98e3-2f5bfbf40356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:22.648529 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a637f0-b2f3-42dd-98e3-2f5bfbf40356 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:22.661047 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:22.721478 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:22.722077 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:22.722355 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:22.722628 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:22.724238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:22.831631 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:22.832975 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe2a2-8bba-4c95-a897-4d46e6084f14 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:22.850267 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 712/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:25.304510 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070017d-b238-4d1e-90ef-38e753724bb2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:25.305080 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070017d-b238-4d1e-90ef-38e753724bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:25.305522 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070017d-b238-4d1e-90ef-38e753724bb2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:25.307629 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:25.313469 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6818-6ec7-47d8-b5d5-594c480d3c8f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:25.313888 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6818-6ec7-47d8-b5d5-594c480d3c8f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:25.314017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e6818-6ec7-47d8-b5d5-594c480d3c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:25.361743 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:25.379728 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ad8e6818-6ec7-47d8-b5d5-594c480d3c8f None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:25.380929 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ad8e6818-6ec7-47d8-b5d5-594c480d3c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:25.477465 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:25.764468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070017d-b238-4d1e-90ef-38e753724bb2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:25.765983 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7070017d-b238-4d1e-90ef-38e753724bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:25.867044 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070017d-b238-4d1e-90ef-38e753724bb2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:25.906022 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:25.939812 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:25.940394 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:25.940667 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:25.940935 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:25.942485 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:26.049340 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:26.050689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d913e4d-75ec-4d1e-8eca-35da62ebb58a service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:26.070012 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 736/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:28.341100 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbaf0-c576-4c8a-af2a-074c6ce9b89e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:28.341661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbaf0-c576-4c8a-af2a-074c6ce9b89e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:28.341661 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbaf0-c576-4c8a-af2a-074c6ce9b89e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:28.342912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:28.348282 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e136b9a-f8a1-4ca0-8b63-bfc02cb854bb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:28.348683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e136b9a-f8a1-4ca0-8b63-bfc02cb854bb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:28.348824 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e136b9a-f8a1-4ca0-8b63-bfc02cb854bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:28.414382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1e136b9a-f8a1-4ca0-8b63-bfc02cb854bb None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:28.415619 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1e136b9a-f8a1-4ca0-8b63-bfc02cb854bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:28.436097 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:28.511576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:28.796499 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cbaf0-c576-4c8a-af2a-074c6ce9b89e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:28.798019 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-033cbaf0-c576-4c8a-af2a-074c6ce9b89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:28.902474 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cbaf0-c576-4c8a-af2a-074c6ce9b89e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:28.923048 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:28.976165 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:28.976762 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:28.977035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:28.977303 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:28.978866 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:29.087662 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:29.089145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d941a2-2daa-4e81-9973-e6826af391e2 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:29.110585 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 715/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:31.310384 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698c561-6ba5-4bf5-a84f-6d416502074c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:31.310946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698c561-6ba5-4bf5-a84f-6d416502074c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:31.310946 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698c561-6ba5-4bf5-a84f-6d416502074c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:31.312299 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:31.317518 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42da20-1581-4ffb-9166-f05db831f91d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:31.317909 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42da20-1581-4ffb-9166-f05db831f91d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:31.318073 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42da20-1581-4ffb-9166-f05db831f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:31.380683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:31.383608 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7e42da20-1581-4ffb-9166-f05db831f91d None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:31.385437 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e42da20-1581-4ffb-9166-f05db831f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:31.481903 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:31.774689 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698c561-6ba5-4bf5-a84f-6d416502074c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:31.776232 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8698c561-6ba5-4bf5-a84f-6d416502074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:31.887089 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698c561-6ba5-4bf5-a84f-6d416502074c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:31.936239 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:31.961556 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:31.962150 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:31.962420 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:31.962683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:31.964289 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:32.070527 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:32.071904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4083e3-ddcc-439e-8020-9c7c7582ba88 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:32.089691 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 739/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:34.306493 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed9e3-9f28-4fb4-9968-6b826f3a6c96 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:34.307017 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed9e3-9f28-4fb4-9968-6b826f3a6c96 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:34.307133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed9e3-9f28-4fb4-9968-6b826f3a6c96 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:34.308470 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:34.313800 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87e811-d76c-4a14-9a0e-8875f2fbc3ca None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:34.314244 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87e811-d76c-4a14-9a0e-8875f2fbc3ca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:34.314373 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87e811-d76c-4a14-9a0e-8875f2fbc3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:34.379052 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bd87e811-d76c-4a14-9a0e-8875f2fbc3ca None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:34.380190 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bd87e811-d76c-4a14-9a0e-8875f2fbc3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:34.412430 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:34.485704 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:34.771915 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ed9e3-9f28-4fb4-9968-6b826f3a6c96 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:34.773473 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-354ed9e3-9f28-4fb4-9968-6b826f3a6c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:34.881966 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ed9e3-9f28-4fb4-9968-6b826f3a6c96 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:34.909536 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:34.955880 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:34.956495 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:34.956775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:34.957050 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:34.958646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:35.067683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:35.069069 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d75d-4d2e-4c43-a415-53f36d8ccafc service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:35.203201 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 718/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:37.408462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06114bd1-ce32-4b81-a9f6-a50a1b07cbd8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:37.408973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06114bd1-ce32-4b81-a9f6-a50a1b07cbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:37.408973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06114bd1-ce32-4b81-a9f6-a50a1b07cbd8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:37.410253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:37.416180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcded4f1-dd7b-4edf-9292-3165f4779b60 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:37.416615 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcded4f1-dd7b-4edf-9292-3165f4779b60 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:37.416773 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcded4f1-dd7b-4edf-9292-3165f4779b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:37.489164 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dcded4f1-dd7b-4edf-9292-3165f4779b60 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:37.491024 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dcded4f1-dd7b-4edf-9292-3165f4779b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:37.499509 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:37.589275 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:37.874635 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06114bd1-ce32-4b81-a9f6-a50a1b07cbd8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:37.876215 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-06114bd1-ce32-4b81-a9f6-a50a1b07cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:37.979973 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06114bd1-ce32-4b81-a9f6-a50a1b07cbd8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:38.037650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:38.056401 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:38.056922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:38.057199 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:38.057463 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:38.059026 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:38.176474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:38.177912 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38f3a-21a4-407e-ba6b-2ffd82c60daf service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:38.196349 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 742/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:40.410186 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61631-35f8-4609-972e-27b3db88900b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:40.410737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61631-35f8-4609-972e-27b3db88900b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:40.410737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61631-35f8-4609-972e-27b3db88900b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:40.412562 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:40.417995 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6b0d-a210-460d-9b57-239f4b141de1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:40.418404 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6b0d-a210-460d-9b57-239f4b141de1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:40.418572 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6b0d-a210-460d-9b57-239f4b141de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:40.483053 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fd9d6b0d-a210-460d-9b57-239f4b141de1 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:40.484265 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fd9d6b0d-a210-460d-9b57-239f4b141de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:40.526425 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:40.589655 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:40.885432 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61631-35f8-4609-972e-27b3db88900b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:40.886990 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a7f61631-35f8-4609-972e-27b3db88900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:40.987943 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61631-35f8-4609-972e-27b3db88900b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:41.025813 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:41.060318 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:41.060927 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:41.061204 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:41.061471 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:41.063065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:41.171394 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:41.172744 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b9647-2143-421a-8920-e4594a84b986 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:41.190065 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 721/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:12:43.355837 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0109d16-e7dc-4c64-99d0-d14bf45ff0c3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:43.356395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0109d16-e7dc-4c64-99d0-d14bf45ff0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:43.356395 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0109d16-e7dc-4c64-99d0-d14bf45ff0c3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:43.357627 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:43.362889 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b80c7d-eb9d-4b15-9a12-92fc3a7d109e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:43.363381 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b80c7d-eb9d-4b15-9a12-92fc3a7d109e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:43.363528 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b80c7d-eb9d-4b15-9a12-92fc3a7d109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:43.427720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c0b80c7d-eb9d-4b15-9a12-92fc3a7d109e None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:43.428926 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c0b80c7d-eb9d-4b15-9a12-92fc3a7d109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:43.446687 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:43.523497 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:43.812465 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0109d16-e7dc-4c64-99d0-d14bf45ff0c3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:43.814024 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0109d16-e7dc-4c64-99d0-d14bf45ff0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:43.916856 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0109d16-e7dc-4c64-99d0-d14bf45ff0c3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:44.097407 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:44.104730 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:44.105331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:44.105608 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:44.105879 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:44.107467 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:44.219565 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:44.220993 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7915b2-13ef-46fc-bf69-87a8e8ff809e service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:44.240419 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 745/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 19:12:46.368300 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580ee1-f0eb-4ddc-9744-3f069d546c8a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:46.368825 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580ee1-f0eb-4ddc-9744-3f069d546c8a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:46.368825 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580ee1-f0eb-4ddc-9744-3f069d546c8a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:46.370085 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 19:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:12:46.375225 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2fbd-5f3e-46ef-8120-aa66da24a426 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:46.375650 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2fbd-5f3e-46ef-8120-aa66da24a426 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:46.375789 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2fbd-5f3e-46ef-8120-aa66da24a426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:46.440085 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2d1a2fbd-5f3e-46ef-8120-aa66da24a426 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:12:46.441277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2d1a2fbd-5f3e-46ef-8120-aa66da24a426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:46.490132 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:46.547432 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 19:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:12:46.829052 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580ee1-f0eb-4ddc-9744-3f069d546c8a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:12:46.830636 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1b580ee1-f0eb-4ddc-9744-3f069d546c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:46.936115 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580ee1-f0eb-4ddc-9744-3f069d546c8a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:12:46.984001 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:47.010746 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:12:47.011382 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:12:47.011667 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:12:47.011939 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:12:47.013526 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:12:47.120855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:12:47.122282 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46301-a622-485d-8cab-483ec25a5dd1 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:12:47.140490 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 724/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 19:12:56.545960 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=22 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:12:57.138262 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=66 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:07.133762 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34d7cd-d9ac-4559-a2bb-d308fbce5346 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:07.134240 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34d7cd-d9ac-4559-a2bb-d308fbce5346 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:07.134353 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34d7cd-d9ac-4559-a2bb-d308fbce5346 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:07.135672 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 747/1471] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:07.140775 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef8249-ec2a-4132-82bd-8072da3a5b08 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:07.141184 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef8249-ec2a-4132-82bd-8072da3a5b08 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:07.141311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef8249-ec2a-4132-82bd-8072da3a5b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:07.205128 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f3ef8249-ec2a-4132-82bd-8072da3a5b08 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:07.206312 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f3ef8249-ec2a-4132-82bd-8072da3a5b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:07.302373 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 725/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:07.606056 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ee49d-3876-465d-a200-91f64d2746d5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:07.606345 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ee49d-3876-465d-a200-91f64d2746d5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:07.606622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ee49d-3876-465d-a200-91f64d2746d5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:07.607975 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 748/1473] 10.222.0.18 () {54 vars in 962 bytes} [Fri Sep 16 19:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:07.612893 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c418a9-3aee-417c-86d3-1c577eaf1cbb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:07.613739 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c418a9-3aee-417c-86d3-1c577eaf1cbb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:07.614043 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c418a9-3aee-417c-86d3-1c577eaf1cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:07.692311 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-19c418a9-3aee-417c-86d3-1c577eaf1cbb None None] MFA Rules not processed for user `e7f6b6c5a3f0462d8da8b4a7afec5335`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:07.693514 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-19c418a9-3aee-417c-86d3-1c577eaf1cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:07.713548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:07.766675 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 726/1474] 10.222.0.18 () {58 vars in 1059 bytes} [Fri Sep 16 19:13:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:07.774904 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ee49d-3876-465d-a200-91f64d2746d5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:07.776411 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c1ee49d-3876-465d-a200-91f64d2746d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:07.879609 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ee49d-3876-465d-a200-91f64d2746d5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:07.954472 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f6b6c5a3f0462d8da8b4a7afec5335', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:07.955070 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:07.955377 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:07.955648 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:07.958277 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:08.010683 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:08.063185 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:08.064559 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f27ca-afda-487f-956a-3c7b12b654d8 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:08.084911 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 749/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:13:10.331378 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfdc43-107e-4ab6-bc01-0dec91c89080 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:10.331916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfdc43-107e-4ab6-bc01-0dec91c89080 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:10.331916 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfdc43-107e-4ab6-bc01-0dec91c89080 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:10.333259 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 727/1476] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:10.338918 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c7c7-1370-40cc-8371-a6605d614162 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:10.339328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c7c7-1370-40cc-8371-a6605d614162 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:10.339503 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4c7c7-1370-40cc-8371-a6605d614162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:10.404705 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9fb4c7c7-1370-40cc-8371-a6605d614162 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:10.405915 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9fb4c7c7-1370-40cc-8371-a6605d614162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:10.499502 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 750/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:10.778540 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfdc43-107e-4ab6-bc01-0dec91c89080 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:10.780107 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fabfdc43-107e-4ab6-bc01-0dec91c89080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:10.902890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfdc43-107e-4ab6-bc01-0dec91c89080 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:10.951098 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:10.994988 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:10.995626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:10.995906 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:10.996180 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:10.996814 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:10.998548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:13:10.998548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 16 19:13:10.998548 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: Sep 16 19:13:10.999723 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adcc3d-359f-4163-bd38-e42bf83e4887 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:11.018412 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 728/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Fri Sep 16 19:13:10 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 19:13:13.175984 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb383ae-e241-45b0-8094-34e2a8072aba None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:13.176597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb383ae-e241-45b0-8094-34e2a8072aba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:13.176597 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb383ae-e241-45b0-8094-34e2a8072aba None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:13.177864 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 751/1479] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:13.183193 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4604a2-5273-4973-aacc-403678b74869 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:13.183650 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4604a2-5273-4973-aacc-403678b74869 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:13.183773 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4604a2-5273-4973-aacc-403678b74869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:13.248353 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8c4604a2-5273-4973-aacc-403678b74869 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:13.249545 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8c4604a2-5273-4973-aacc-403678b74869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:13.343658 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 729/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:13.643306 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb383ae-e241-45b0-8094-34e2a8072aba None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:13.645489 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cfb383ae-e241-45b0-8094-34e2a8072aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:13.655445 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:13.795907 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb383ae-e241-45b0-8094-34e2a8072aba None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:13.874291 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:13.906493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d7573ee32f6443892a361f012c2f204', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:13.907008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:13.907314 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:13.907576 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:13.909137 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:13.996129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:13.997725 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411a344-68c0-41bb-895d-3b85cdce3910 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:14.015429 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 752/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:13:16.368335 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189d2d-28c0-41db-a8c0-985c4d9b4c2f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:16.368858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189d2d-28c0-41db-a8c0-985c4d9b4c2f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:16.368858 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189d2d-28c0-41db-a8c0-985c4d9b4c2f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:16.370156 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 730/1482] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:16.375474 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb2cfe-e213-4dce-bfd7-dfe711b86753 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:16.375931 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb2cfe-e213-4dce-bfd7-dfe711b86753 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:16.376039 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb2cfe-e213-4dce-bfd7-dfe711b86753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:16.439591 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4fdb2cfe-e213-4dce-bfd7-dfe711b86753 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:16.441280 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4fdb2cfe-e213-4dce-bfd7-dfe711b86753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:16.486769 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:16.537179 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 753/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:16.828025 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0189d2d-28c0-41db-a8c0-985c4d9b4c2f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:16.829557 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c0189d2d-28c0-41db-a8c0-985c4d9b4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:16.853741 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:16.932877 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0189d2d-28c0-41db-a8c0-985c4d9b4c2f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:17.005074 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:17.005678 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:17.005958 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:17.006232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:17.007848 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:17.084804 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:17.119993 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:17.121465 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27676-71a8-4341-a37f-2113ad751012 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:17.139536 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 731/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:13:19.372961 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e49d3a-774e-41da-8835-47b05248421b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:19.373501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e49d3a-774e-41da-8835-47b05248421b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:19.373501 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e49d3a-774e-41da-8835-47b05248421b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:19.374760 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 754/1485] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:19.380596 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae6cdb-04b0-4cee-8b36-e10df05a05b4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:19.381032 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae6cdb-04b0-4cee-8b36-e10df05a05b4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:19.381138 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae6cdb-04b0-4cee-8b36-e10df05a05b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:19.445606 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f6ae6cdb-04b0-4cee-8b36-e10df05a05b4 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:19.446810 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f6ae6cdb-04b0-4cee-8b36-e10df05a05b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:19.541386 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 732/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:19.839807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e49d3a-774e-41da-8835-47b05248421b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:19.841309 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-21e49d3a-774e-41da-8835-47b05248421b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:19.960090 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e49d3a-774e-41da-8835-47b05248421b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:20.031941 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:20.033606 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:20.034448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:20.034920 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:20.035217 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:20.037508 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:20.145045 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:20.146392 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e29b4-eeee-4fd4-b45e-3e75f0c66bfb service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:20.163835 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 755/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:13:22.486238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc3c62-72e0-4825-ae96-1d9dd4e29319 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:22.486807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc3c62-72e0-4825-ae96-1d9dd4e29319 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:22.486807 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc3c62-72e0-4825-ae96-1d9dd4e29319 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:22.488486 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 733/1488] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:22.493771 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f5637-f5a1-479c-bcc9-7cb97005d379 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:22.494172 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f5637-f5a1-479c-bcc9-7cb97005d379 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:22.494326 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f5637-f5a1-479c-bcc9-7cb97005d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:22.559759 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-362f5637-f5a1-479c-bcc9-7cb97005d379 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:22.560958 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-362f5637-f5a1-479c-bcc9-7cb97005d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:22.613255 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:22.656770 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 756/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:22.951366 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc3c62-72e0-4825-ae96-1d9dd4e29319 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:22.952961 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d1dc3c62-72e0-4825-ae96-1d9dd4e29319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:22.988829 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:23.055869 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc3c62-72e0-4825-ae96-1d9dd4e29319 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:23.129626 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:23.130235 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:23.130502 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:23.130782 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:23.132406 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:23.211446 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:23.239587 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:23.241096 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932bc0e-0b67-4f24-bc4d-6e934ee4af10 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:23.258737 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 734/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:13:25.602248 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8029f-9c1f-4238-b1b5-1df4800ee5cf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:25.602813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8029f-9c1f-4238-b1b5-1df4800ee5cf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:25.602813 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8029f-9c1f-4238-b1b5-1df4800ee5cf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:25.604249 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 757/1491] 10.222.0.18 () {54 vars in 927 bytes} [Fri Sep 16 19:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:13:25.609629 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e7ef6-dfa0-4264-8154-267f134efa29 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:25.610068 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e7ef6-dfa0-4264-8154-267f134efa29 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:25.610185 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e7ef6-dfa0-4264-8154-267f134efa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:25.675750 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-198e7ef6-dfa0-4264-8154-267f134efa29 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:13:25.676953 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-198e7ef6-dfa0-4264-8154-267f134efa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:25.771173 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 735/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Fri Sep 16 19:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:13:26.062579 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8029f-9c1f-4238-b1b5-1df4800ee5cf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:13:26.064125 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9a8029f-9c1f-4238-b1b5-1df4800ee5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:26.164468 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8029f-9c1f-4238-b1b5-1df4800ee5cf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:13:26.236482 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:13:26.237080 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:13:26.237358 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:13:26.237628 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:13:26.239486 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:13:26.247717 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:26.346682 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:13:26.348073 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29d5fc-4269-4af3-a0c7-99e740457c38 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:13:26.366333 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 758/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:13:35.769659 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=82 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:13:36.364190 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=50 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:46.633071 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:46.633646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:46.633646 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:46.706933 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:14:46.708158 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:46.801599 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 736/1494] 10.222.0.18 () {58 vars in 958 bytes} [Fri Sep 16 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:14:46.809618 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:46.810110 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:46.810232 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:46.884328 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:14:46.885441 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:46.977018 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 759/1495] 10.222.0.18 () {58 vars in 958 bytes} [Fri Sep 16 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:14:46.981722 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:46.982947 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:46.994840 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ffb1-f7b9-4de3-8264-878ba6f853e7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:47.053855 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:47.088803 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:47.089410 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.089685 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.089954 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:47.090581 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 19:14:47.090875 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:47.091905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:47.091905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 16 19:14:47.091905 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: Sep 16 19:14:47.092777 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:47.103751 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 737/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Fri Sep 16 19:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 19:14:47.112819 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:47.121692 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a056-e1ad-4674-a139-9a8f7447c7c4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:47.180890 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:47.213246 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:47.213830 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.214103 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.214369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:47.214988 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:47.216715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:47.216715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 16 19:14:47.216715 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: Sep 16 19:14:47.217885 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:47.274263 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 760/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Fri Sep 16 19:14:47 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 19:14:47.278617 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:47.280222 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:47.314965 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dafc2-c950-4384-a294-6ca8351ce844 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:47.416076 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:47.416692 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.416970 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.417239 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:47.417912 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:47.418890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:47.418890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 16 19:14:47.418890 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: Sep 16 19:14:47.420123 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:47.460897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 738/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Fri Sep 16 19:14:47 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 19:14:47.464953 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:47.466427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:47.526601 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb0f0-5ec9-41dc-8fee-5f538be6cf7d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:47.611406 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:47.622519 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:47.623129 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.623450 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.623728 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:47.624368 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:47.625382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:47.625382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 16 19:14:47.625382 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: Sep 16 19:14:47.626268 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:47.638331 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 761/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Sep 16 19:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 19:14:47.643106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:47.644668 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:47.655829 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125be00-6da7-4909-9c4c-c407f5eb9690 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:47.680632 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:47.749221 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:47.749897 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.750263 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.750538 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:47.751208 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:47.752234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:47.752234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 16 19:14:47.752234 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: Sep 16 19:14:47.753106 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5e853-db4f-4a73-818c-77be19008088 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:47.765145 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 739/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Fri Sep 16 19:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 19:14:47.769891 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:47.771448 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:47.804722 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6691b-eaa4-4f5c-a078-563832d36e14 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:47.897477 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:47.898127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.898402 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.898669 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] PATH_INFO: `/v3/projects/0c09787693ac454c86fb272e957fd854/users/46a6fcc3754f4f738c27cbf47e722c78/roles/f2e63848ed864ae5901749a96467215f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:47.920796 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c09787693ac454c86fb272e957fd854, user_id=46a6fcc3754f4f738c27cbf47e722c78, role_id=f2e63848ed864ae5901749a96467215f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:47.921811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:47.921811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 16 19:14:47.921811 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: Sep 16 19:14:47.923807 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f50d5-8f9f-4364-9860-2675b8aa3ee7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:47.947493 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 762/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Fri Sep 16 19:14:47 2022] PUT /identity/v3/projects/0c09787693ac454c86fb272e957fd854/users/46a6fcc3754f4f738c27cbf47e722c78/roles/f2e63848ed864ae5901749a96467215f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:14:47.960480 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c154a-224d-458f-b1cb-489ee5fc3e12 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:47.960730 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c154a-224d-458f-b1cb-489ee5fc3e12 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:47.961008 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c154a-224d-458f-b1cb-489ee5fc3e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:48.017785 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-659c154a-224d-458f-b1cb-489ee5fc3e12 None None] MFA Rules not processed for user `46a6fcc3754f4f738c27cbf47e722c78`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:14:48.018898 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-659c154a-224d-458f-b1cb-489ee5fc3e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:48.096088 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 740/1502] 10.222.0.18 () {58 vars in 958 bytes} [Fri Sep 16 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:14:48.109170 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71210c41-ac28-48a1-876c-ebd6bebb351b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:48.109462 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71210c41-ac28-48a1-876c-ebd6bebb351b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:48.109742 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71210c41-ac28-48a1-876c-ebd6bebb351b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:48.111385 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 763/1503] 10.222.0.18 () {54 vars in 964 bytes} [Fri Sep 16 19:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:14:48.116035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca4878-5160-4d1d-95f9-f673d38b442b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:48.116325 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca4878-5160-4d1d-95f9-f673d38b442b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:48.116605 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca4878-5160-4d1d-95f9-f673d38b442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:48.148597 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:48.193772 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5aca4878-5160-4d1d-95f9-f673d38b442b None None] MFA Rules not processed for user `0b094a1d708f4b02b83405e6befbed9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:14:48.194970 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5aca4878-5160-4d1d-95f9-f673d38b442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:48.269125 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 741/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Fri Sep 16 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:14:48.278657 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71210c41-ac28-48a1-876c-ebd6bebb351b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:48.280270 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-71210c41-ac28-48a1-876c-ebd6bebb351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:48.333518 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:48.387998 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71210c41-ac28-48a1-876c-ebd6bebb351b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:48.460210 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b094a1d708f4b02b83405e6befbed9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:48.460802 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:48.461074 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:48.461338 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:48.462889 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:48.557281 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:48.558622 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:48.561999 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:48.577439 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 764/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:14:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:14:48.622083 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475c660-72b9-43da-83d3-9293c4ca7c89 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:48.622372 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475c660-72b9-43da-83d3-9293c4ca7c89 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:48.622643 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475c660-72b9-43da-83d3-9293c4ca7c89 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:48.623921 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 742/1506] 10.222.0.18 () {52 vars in 830 bytes} [Fri Sep 16 19:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 19:14:48.628071 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:48.629278 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:48.724454 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330a9c9-d6ee-4eb3-9e45-3994ad042555 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:48.815051 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:48.815633 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:48.815909 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:48.816177 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] PATH_INFO: `/v3/users/46a6fcc3754f4f738c27cbf47e722c78` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:48.828850 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a6fcc3754f4f738c27cbf47e722c78)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:48.829897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:48.829897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 16 19:14:48.829897 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: Sep 16 19:14:48.831102 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:48.840649 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:48.898262 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a6fcc3754f4f738c27cbf47e722c78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:48.898912 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a6fcc3754f4f738c27cbf47e722c78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:48.899200 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a6fcc3754f4f738c27cbf47e722c78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:48.914922 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a6fcc3754f4f738c27cbf47e722c78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:48.937427 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a6fcc3754f4f738c27cbf47e722c78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:48.952253 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 765/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Fri Sep 16 19:14:48 2022] DELETE /identity/v3/users/46a6fcc3754f4f738c27cbf47e722c78 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:14:48.956742 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:48.957477 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:48.957674 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:49.014328 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] MFA Rules not processed for user `153cf9ce284c4a19b0247d5e1bc68384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 19:14:49.015586 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:49.089261 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:49.102236 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 743/1508] 10.222.0.18 () {58 vars in 958 bytes} [Fri Sep 16 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 19:14:49.112691 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:49.113917 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:49.211298 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761acf-cc37-4d42-9c80-ac31d9baafcf admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:49.263550 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:49.285369 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc1c6323c0a34d8f9d36fdc95712d26f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad877dffc51743268f5541983c925a8d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:49.285972 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:49.286252 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:49.286526 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:49.288127 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:49.393663 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:49.395008 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7f9a-7b0b-431f-bebf-7d192466c721 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:49.413151 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 766/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Fri Sep 16 19:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 19:14:49.736568 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 19:14:49.737794 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 19:14:49.749454 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cdf17-9117-4d06-8b81-e50cb23e3b94 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 19:14:49.847232 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '153cf9ce284c4a19b0247d5e1bc68384', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a6c3a4a3b2c47b590674deda9305a68', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 19:14:49.848007 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 19:14:49.848238 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 19:14:49.849331 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] PATH_INFO: `/v3/projects/0c09787693ac454c86fb272e957fd854` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 19:14:49.859104 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c09787693ac454c86fb272e957fd854)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 19:14:49.860683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 19:14:49.860683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 16 19:14:49.860683 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: Sep 16 19:14:49.862133 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 19:14:49.918312 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c09787693ac454c86fb272e957fd854', 'request_id': 'req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:49.934035 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c09787693ac454c86fb272e957fd854', 'request_id': 'req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:49.944327 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c09787693ac454c86fb272e957fd854', 'request_id': 'req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:49.967996 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] The token cache is being invalidate because project 0c09787693ac454c86fb272e957fd854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 19:14:49.968726 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:49.968826 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fef8812b-52c3-46d9-be11-f9d9ea5b58bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 19:14:49.970345 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 744/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Fri Sep 16 19:14:49 2022] DELETE /identity/v3/projects/0c09787693ac454c86fb272e957fd854 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 19:14:59.411066 dv-cinder-803773-5-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=62 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 19:14:59.965720 dv-cinder-803773-5-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}